2026-02-22T23:10:51.624 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-22T23:10:51.650 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T23:10:51.743 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361 branch: wip-yuri6-testing-2026-02-22-1629-squid description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '64361' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri6-testing-2026-02-22-1629-squid ceph: conf: client: debug monc: 20 debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async osd heartbeat grace: 160 osd op thread timeout: 150 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 mon warn on pool no app: false paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd class default list: '*' osd class load list: '*' osd client watch timeout: 240 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - reached quota - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(SLOW_OPS\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(OBJECT_MISPLACED\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request - OSD bench result sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 stop-daemons-timeout: 1200 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2026-02-22-1629-squid sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 5907 sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2026-02-22-1629-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 targets: trial006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHtq4id0Rnt4Dlue3y07cxPgX+ypWyWEsAZ6LPQ+y660ESLNEaR1PP/DxoDSDNGaneZOP6cNPTyatKww8WF16q0= trial040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXHopXURMt5cdZ5VAgzVR3xz7qzxurn1Fy31EK4ll2icR0n21w2nPGqlQYz6wk1ecv63AfkJEgCUyPI26uxGYQ= trial080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOHh1jaXgsyBoMa05cVAm3f5zIYMzDtEBvSW3S2p6YZju4hFoH+f5Jdwr0/ap6UVdGqNtsyxijREtEAE1RnSKK4= trial189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOzk7PS1aiuAjw0M5FrfAHxbpnMP1CuoP60ehFTcObSPhZrIWiSvzG1VMUIobglYgYW/1bKvxZoJgTZmzskRsAc= tasks: - install: null - ceph: null - workunit: clients: client.0: - rados/test.sh env: ALLOW_TIMEOUTS: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - '-- Don''t run this test on tiny clusters if py_len(yaml.roles) < 4 then reject() end ' teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-22_19:35:40 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-22T23:10:51.743 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa; will attempt to use it 2026-02-22T23:10:51.743 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks 2026-02-22T23:10:51.744 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-22T23:10:51.745 INFO:teuthology.task.internal:Checking packages... 2026-02-22T23:10:51.746 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f90f4eaebb28b619cf76e80cbbe997656da69579' 2026-02-22T23:10:51.746 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-22T23:10:51.746 INFO:teuthology.packaging:ref: None 2026-02-22T23:10:51.746 INFO:teuthology.packaging:tag: None 2026-02-22T23:10:51.746 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:10:51.746 INFO:teuthology.packaging:sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:10:51.746 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:10:51.866 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-689.gf90f4eae 2026-02-22T23:10:51.868 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-22T23:10:51.880 INFO:teuthology.task.internal:no buildpackages task found 2026-02-22T23:10:51.880 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-22T23:10:51.893 INFO:teuthology.task.internal:Saving configuration 2026-02-22T23:10:51.900 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-22T23:10:51.911 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-22T23:10:51.987 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial006.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 23:07:18.490279', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHtq4id0Rnt4Dlue3y07cxPgX+ypWyWEsAZ6LPQ+y660ESLNEaR1PP/DxoDSDNGaneZOP6cNPTyatKww8WF16q0='} 2026-02-22T23:10:52.043 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial040.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 23:07:18.489862', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXHopXURMt5cdZ5VAgzVR3xz7qzxurn1Fy31EK4ll2icR0n21w2nPGqlQYz6wk1ecv63AfkJEgCUyPI26uxGYQ='} 2026-02-22T23:10:52.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial080.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 23:07:18.489243', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOHh1jaXgsyBoMa05cVAm3f5zIYMzDtEBvSW3S2p6YZju4hFoH+f5Jdwr0/ap6UVdGqNtsyxijREtEAE1RnSKK4='} 2026-02-22T23:10:52.161 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial189.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 23:07:18.490656', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOzk7PS1aiuAjw0M5FrfAHxbpnMP1CuoP60ehFTcObSPhZrIWiSvzG1VMUIobglYgYW/1bKvxZoJgTZmzskRsAc='} 2026-02-22T23:10:52.161 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-22T23:10:52.172 INFO:teuthology.task.internal:roles: ubuntu@trial006.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2026-02-22T23:10:52.172 INFO:teuthology.task.internal:roles: ubuntu@trial040.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2026-02-22T23:10:52.172 INFO:teuthology.task.internal:roles: ubuntu@trial080.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2026-02-22T23:10:52.172 INFO:teuthology.task.internal:roles: ubuntu@trial189.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2026-02-22T23:10:52.172 INFO:teuthology.run_tasks:Running task console_log... 2026-02-22T23:10:52.437 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x75d82f21fec0>, signals=[15]) 2026-02-22T23:10:52.437 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-22T23:10:52.448 INFO:teuthology.task.internal:Opening connections... 2026-02-22T23:10:52.448 DEBUG:teuthology.task.internal:connecting to ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:10:52.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:10:52.512 DEBUG:teuthology.task.internal:connecting to ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:10:52.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:10:52.577 DEBUG:teuthology.task.internal:connecting to ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:10:52.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:10:52.637 DEBUG:teuthology.task.internal:connecting to ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:10:52.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:10:52.702 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-22T23:10:52.714 DEBUG:teuthology.orchestra.run.trial006:> uname -m 2026-02-22T23:10:52.731 INFO:teuthology.orchestra.run.trial006.stdout:x86_64 2026-02-22T23:10:52.731 DEBUG:teuthology.orchestra.run.trial006:> cat /etc/os-release 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:NAME="CentOS Stream" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:VERSION="9" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:ID="centos" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:ID_LIKE="rhel fedora" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:VERSION_ID="9" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:PLATFORM_ID="platform:el9" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:ANSI_COLOR="0;31" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:LOGO="fedora-logo-icon" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:HOME_URL="https://centos.org/" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-22T23:10:52.786 INFO:teuthology.orchestra.run.trial006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-22T23:10:52.787 INFO:teuthology.lock.ops:Updating trial006.front.sepia.ceph.com on lock server 2026-02-22T23:10:52.857 DEBUG:teuthology.orchestra.run.trial040:> uname -m 2026-02-22T23:10:52.872 INFO:teuthology.orchestra.run.trial040.stdout:x86_64 2026-02-22T23:10:52.873 DEBUG:teuthology.orchestra.run.trial040:> cat /etc/os-release 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:NAME="CentOS Stream" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:VERSION="9" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:ID="centos" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:ID_LIKE="rhel fedora" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:VERSION_ID="9" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:PLATFORM_ID="platform:el9" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:ANSI_COLOR="0;31" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:LOGO="fedora-logo-icon" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-22T23:10:52.929 INFO:teuthology.orchestra.run.trial040.stdout:HOME_URL="https://centos.org/" 2026-02-22T23:10:52.930 INFO:teuthology.orchestra.run.trial040.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-22T23:10:52.930 INFO:teuthology.orchestra.run.trial040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-22T23:10:52.930 INFO:teuthology.orchestra.run.trial040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-22T23:10:52.930 INFO:teuthology.lock.ops:Updating trial040.front.sepia.ceph.com on lock server 2026-02-22T23:10:53.006 DEBUG:teuthology.orchestra.run.trial080:> uname -m 2026-02-22T23:10:53.022 INFO:teuthology.orchestra.run.trial080.stdout:x86_64 2026-02-22T23:10:53.022 DEBUG:teuthology.orchestra.run.trial080:> cat /etc/os-release 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:NAME="CentOS Stream" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:VERSION="9" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:ID="centos" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:ID_LIKE="rhel fedora" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:VERSION_ID="9" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:PLATFORM_ID="platform:el9" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:ANSI_COLOR="0;31" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:LOGO="fedora-logo-icon" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:HOME_URL="https://centos.org/" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-22T23:10:53.078 INFO:teuthology.orchestra.run.trial080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-22T23:10:53.079 INFO:teuthology.orchestra.run.trial080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-22T23:10:53.079 INFO:teuthology.lock.ops:Updating trial080.front.sepia.ceph.com on lock server 2026-02-22T23:10:53.155 DEBUG:teuthology.orchestra.run.trial189:> uname -m 2026-02-22T23:10:53.171 INFO:teuthology.orchestra.run.trial189.stdout:x86_64 2026-02-22T23:10:53.171 DEBUG:teuthology.orchestra.run.trial189:> cat /etc/os-release 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:NAME="CentOS Stream" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:VERSION="9" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:ID="centos" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:ID_LIKE="rhel fedora" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:VERSION_ID="9" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:PLATFORM_ID="platform:el9" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:ANSI_COLOR="0;31" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:LOGO="fedora-logo-icon" 2026-02-22T23:10:53.227 INFO:teuthology.orchestra.run.trial189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-22T23:10:53.228 INFO:teuthology.orchestra.run.trial189.stdout:HOME_URL="https://centos.org/" 2026-02-22T23:10:53.228 INFO:teuthology.orchestra.run.trial189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-22T23:10:53.228 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-22T23:10:53.228 INFO:teuthology.orchestra.run.trial189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-22T23:10:53.228 INFO:teuthology.lock.ops:Updating trial189.front.sepia.ceph.com on lock server 2026-02-22T23:10:53.303 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-22T23:10:53.319 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-22T23:10:53.330 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-22T23:10:53.331 DEBUG:teuthology.orchestra.run.trial006:> test '!' -e /home/ubuntu/cephtest 2026-02-22T23:10:53.332 DEBUG:teuthology.orchestra.run.trial040:> test '!' -e /home/ubuntu/cephtest 2026-02-22T23:10:53.334 DEBUG:teuthology.orchestra.run.trial080:> test '!' -e /home/ubuntu/cephtest 2026-02-22T23:10:53.335 DEBUG:teuthology.orchestra.run.trial189:> test '!' -e /home/ubuntu/cephtest 2026-02-22T23:10:53.349 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-22T23:10:53.361 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-22T23:10:53.362 DEBUG:teuthology.orchestra.run.trial006:> test -z $(ls -A /var/lib/ceph) 2026-02-22T23:10:53.389 DEBUG:teuthology.orchestra.run.trial040:> test -z $(ls -A /var/lib/ceph) 2026-02-22T23:10:53.390 DEBUG:teuthology.orchestra.run.trial080:> test -z $(ls -A /var/lib/ceph) 2026-02-22T23:10:53.392 DEBUG:teuthology.orchestra.run.trial189:> test -z $(ls -A /var/lib/ceph) 2026-02-22T23:10:53.408 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-22T23:10:53.664 INFO:teuthology.run_tasks:Running task kernel... 2026-02-22T23:10:53.686 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-22T23:10:53.686 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mon.b': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mon.c': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-22T23:10:53.686 DEBUG:teuthology.orchestra.run.trial006:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T23:10:53.686 DEBUG:teuthology.orchestra.run.trial040:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T23:10:53.686 DEBUG:teuthology.orchestra.run.trial080:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T23:10:53.687 DEBUG:teuthology.orchestra.run.trial189:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T23:10:53.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:10:53.701 DEBUG:teuthology.orchestra.run.trial040:> uname -r 2026-02-22T23:10:53.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:10:53.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:10:53.702 DEBUG:teuthology.orchestra.run.trial189:> uname -r 2026-02-22T23:10:53.702 DEBUG:teuthology.orchestra.run.trial080:> uname -r 2026-02-22T23:10:53.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:10:53.703 DEBUG:teuthology.orchestra.run.trial006:> uname -r 2026-02-22T23:10:53.757 INFO:teuthology.orchestra.run.trial040.stdout:5.14.0-681.el9.x86_64 2026-02-22T23:10:53.757 INFO:teuthology.task.kernel:Running kernel on trial040: 5.14.0-681.el9.x86_64 2026-02-22T23:10:53.757 DEBUG:teuthology.orchestra.run.trial040:> sudo yum install -y kernel 2026-02-22T23:10:53.757 INFO:teuthology.orchestra.run.trial080.stdout:5.14.0-681.el9.x86_64 2026-02-22T23:10:53.757 INFO:teuthology.task.kernel:Running kernel on trial080: 5.14.0-681.el9.x86_64 2026-02-22T23:10:53.758 DEBUG:teuthology.orchestra.run.trial080:> sudo yum install -y kernel 2026-02-22T23:10:53.758 INFO:teuthology.orchestra.run.trial189.stdout:5.14.0-681.el9.x86_64 2026-02-22T23:10:53.758 INFO:teuthology.task.kernel:Running kernel on trial189: 5.14.0-681.el9.x86_64 2026-02-22T23:10:53.758 DEBUG:teuthology.orchestra.run.trial189:> sudo yum install -y kernel 2026-02-22T23:10:53.759 INFO:teuthology.orchestra.run.trial006.stdout:5.14.0-681.el9.x86_64 2026-02-22T23:10:53.759 INFO:teuthology.task.kernel:Running kernel on trial006: 5.14.0-681.el9.x86_64 2026-02-22T23:10:53.759 DEBUG:teuthology.orchestra.run.trial006:> sudo yum install -y kernel 2026-02-22T23:10:54.849 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-22T23:10:54.976 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-22T23:10:55.732 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - AppStream 75 MB/s | 27 MB 00:00 2026-02-22T23:10:55.838 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.9 MB 00:01 2026-02-22T23:10:58.052 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-22T23:10:59.042 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-22T23:10:59.150 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 7.8 MB 00:01 2026-02-22T23:11:00.293 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 20 kB 00:00 2026-02-22T23:11:00.459 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 8.9 MB 00:06 2026-02-22T23:11:01.055 INFO:teuthology.orchestra.run.trial189.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-22T23:11:02.301 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 7.8 MB 00:00 2026-02-22T23:11:02.516 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 7.8 MB 00:02 2026-02-22T23:11:03.157 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - Extras packages 140 kB/s | 20 kB 00:00 2026-02-22T23:11:03.480 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-22T23:11:03.549 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 20 kB 00:00 2026-02-22T23:11:04.196 INFO:teuthology.orchestra.run.trial189.stdout:lab-extras 2.0 MB/s | 50 kB 00:00 2026-02-22T23:11:04.727 INFO:teuthology.orchestra.run.trial006.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2026-02-22T23:11:05.210 INFO:teuthology.orchestra.run.trial189.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-22T23:11:05.210 INFO:teuthology.orchestra.run.trial189.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-22T23:11:05.229 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:11:05.229 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:11:05.229 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:11:05.253 DEBUG:teuthology.orchestra.run.trial189:> echo no | sudo yum reinstall kernel || true 2026-02-22T23:11:05.554 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:01 ago on Sun 22 Feb 2026 11:11:04 PM UTC. 2026-02-22T23:11:05.634 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout:Reinstalling: 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:11:05.635 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 803 k 2026-02-22T23:11:05.636 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 0 2026-02-22T23:11:05.636 INFO:teuthology.orchestra.run.trial189.stderr:Operation aborted. 2026-02-22T23:11:05.656 INFO:teuthology.orchestra.run.trial189.stdout:Is this ok [y/N]: 2026-02-22T23:11:05.656 DEBUG:teuthology.orchestra.run.trial189:> sudo yum reinstall -y kernel || true 2026-02-22T23:11:05.954 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:01 ago on Sun 22 Feb 2026 11:11:04 PM UTC. 2026-02-22T23:11:06.034 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout:Reinstalling: 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:11:06.035 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 803 k 2026-02-22T23:11:06.036 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 0 2026-02-22T23:11:06.036 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:11:06.253 INFO:teuthology.orchestra.run.trial189.stdout:kernel-5.14.0-681.el9.x86_64.rpm 6.1 MB/s | 803 kB 00:00 2026-02-22T23:11:06.254 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:11:06.254 INFO:teuthology.orchestra.run.trial189.stdout:Total 3.6 MB/s | 803 kB 00:00 2026-02-22T23:11:06.264 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:11:06.297 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:11:06.297 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:11:06.301 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:11:06.301 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:11:06.325 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:11:06.329 INFO:teuthology.orchestra.run.trial189.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:06.330 INFO:teuthology.orchestra.run.trial189.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:06.442 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:06.443 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:06.496 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:06.497 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:11:06.497 INFO:teuthology.orchestra.run.trial189.stdout:Reinstalled: 2026-02-22T23:11:06.497 INFO:teuthology.orchestra.run.trial189.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:06.497 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:11:06.497 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:11:06.525 DEBUG:teuthology.orchestra.run.trial189:> rpm -q kernel | sort -rV | head -n 1 2026-02-22T23:11:06.550 INFO:teuthology.orchestra.run.trial189.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:06.551 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-22T23:11:06.551 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T23:11:06.551 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-22T23:11:06.551 DEBUG:teuthology.orchestra.run.trial189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T23:11:06.615 INFO:teuthology.orchestra.run.trial189.stdout:ttyS1 2026-02-22T23:11:06.636 DEBUG:teuthology.parallel:result is None 2026-02-22T23:11:07.027 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 7.8 MB 00:01 2026-02-22T23:11:07.878 INFO:teuthology.orchestra.run.trial006.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-22T23:11:07.880 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - Extras packages 156 kB/s | 20 kB 00:00 2026-02-22T23:11:08.889 INFO:teuthology.orchestra.run.trial006.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-22T23:11:08.889 INFO:teuthology.orchestra.run.trial006.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-22T23:11:08.909 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:11:08.910 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:11:08.910 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:11:08.934 DEBUG:teuthology.orchestra.run.trial006:> echo no | sudo yum reinstall kernel || true 2026-02-22T23:11:09.231 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:02 ago on Sun 22 Feb 2026 11:11:07 PM UTC. 2026-02-22T23:11:09.313 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout:Reinstalling: 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 803 k 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 0 2026-02-22T23:11:09.314 INFO:teuthology.orchestra.run.trial006.stderr:Operation aborted. 2026-02-22T23:11:09.334 INFO:teuthology.orchestra.run.trial006.stdout:Is this ok [y/N]: 2026-02-22T23:11:09.334 DEBUG:teuthology.orchestra.run.trial006:> sudo yum reinstall -y kernel || true 2026-02-22T23:11:09.633 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:02 ago on Sun 22 Feb 2026 11:11:07 PM UTC. 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout:Reinstalling: 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:11:09.714 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:11:09.715 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:11:09.715 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 803 k 2026-02-22T23:11:09.715 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 0 2026-02-22T23:11:09.715 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:11:09.969 INFO:teuthology.orchestra.run.trial006.stdout:kernel-5.14.0-681.el9.x86_64.rpm 4.9 MB/s | 803 kB 00:00 2026-02-22T23:11:09.969 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:11:09.969 INFO:teuthology.orchestra.run.trial006.stdout:Total 3.1 MB/s | 803 kB 00:00 2026-02-22T23:11:09.979 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:11:10.013 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:11:10.013 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:11:10.017 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:11:10.018 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:11:10.041 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:11:10.045 INFO:teuthology.orchestra.run.trial006.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:10.045 INFO:teuthology.orchestra.run.trial006.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:10.154 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:10.154 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:10.209 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:10.209 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:11:10.209 INFO:teuthology.orchestra.run.trial006.stdout:Reinstalled: 2026-02-22T23:11:10.209 INFO:teuthology.orchestra.run.trial006.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:10.209 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:11:10.210 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:11:10.237 DEBUG:teuthology.orchestra.run.trial006:> rpm -q kernel | sort -rV | head -n 1 2026-02-22T23:11:10.260 INFO:teuthology.orchestra.run.trial006.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:10.260 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-22T23:11:10.260 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T23:11:10.260 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-22T23:11:10.260 DEBUG:teuthology.orchestra.run.trial006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T23:11:10.325 INFO:teuthology.orchestra.run.trial006.stdout:ttyS1 2026-02-22T23:11:10.355 DEBUG:teuthology.parallel:result is None 2026-02-22T23:11:15.075 INFO:teuthology.orchestra.run.trial040.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 20 MB 00:11 2026-02-22T23:11:18.244 INFO:teuthology.orchestra.run.trial040.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-22T23:11:19.256 INFO:teuthology.orchestra.run.trial040.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-22T23:11:19.256 INFO:teuthology.orchestra.run.trial040.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-22T23:11:19.275 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:11:19.276 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:11:19.276 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:11:19.298 DEBUG:teuthology.orchestra.run.trial040:> echo no | sudo yum reinstall kernel || true 2026-02-22T23:11:19.644 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:01 ago on Sun 22 Feb 2026 11:11:18 PM UTC. 2026-02-22T23:11:19.726 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout:Reinstalling: 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:11:19.727 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 803 k 2026-02-22T23:11:19.728 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 0 2026-02-22T23:11:19.728 INFO:teuthology.orchestra.run.trial040.stderr:Operation aborted. 2026-02-22T23:11:19.747 INFO:teuthology.orchestra.run.trial040.stdout:Is this ok [y/N]: 2026-02-22T23:11:19.748 DEBUG:teuthology.orchestra.run.trial040:> sudo yum reinstall -y kernel || true 2026-02-22T23:11:19.818 INFO:teuthology.orchestra.run.trial080.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 20 MB 00:11 2026-02-22T23:11:20.050 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:02 ago on Sun 22 Feb 2026 11:11:18 PM UTC. 2026-02-22T23:11:20.132 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:Reinstalling: 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 803 k 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 0 2026-02-22T23:11:20.133 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:11:21.103 INFO:teuthology.orchestra.run.trial040.stdout:kernel-5.14.0-681.el9.x86_64.rpm 885 kB/s | 803 kB 00:00 2026-02-22T23:11:21.103 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:11:21.103 INFO:teuthology.orchestra.run.trial040.stdout:Total 828 kB/s | 803 kB 00:00 2026-02-22T23:11:21.112 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:11:21.147 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:11:21.147 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:11:21.151 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:11:21.151 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:11:21.175 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:11:21.179 INFO:teuthology.orchestra.run.trial040.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:21.179 INFO:teuthology.orchestra.run.trial040.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:21.291 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:21.291 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:21.347 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:21.348 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:11:21.348 INFO:teuthology.orchestra.run.trial040.stdout:Reinstalled: 2026-02-22T23:11:21.348 INFO:teuthology.orchestra.run.trial040.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:21.348 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:11:21.348 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:11:21.376 DEBUG:teuthology.orchestra.run.trial040:> rpm -q kernel | sort -rV | head -n 1 2026-02-22T23:11:21.401 INFO:teuthology.orchestra.run.trial040.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:21.402 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-22T23:11:21.402 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T23:11:21.402 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2026-02-22T23:11:21.402 DEBUG:teuthology.orchestra.run.trial040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T23:11:21.467 INFO:teuthology.orchestra.run.trial040.stdout:ttyS1 2026-02-22T23:11:21.495 DEBUG:teuthology.parallel:result is None 2026-02-22T23:11:22.945 INFO:teuthology.orchestra.run.trial080.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-22T23:11:23.957 INFO:teuthology.orchestra.run.trial080.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-22T23:11:23.957 INFO:teuthology.orchestra.run.trial080.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-22T23:11:23.977 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:11:23.978 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:11:23.978 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:11:24.001 DEBUG:teuthology.orchestra.run.trial080:> echo no | sudo yum reinstall kernel || true 2026-02-22T23:11:24.302 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:02 ago on Sun 22 Feb 2026 11:11:22 PM UTC. 2026-02-22T23:11:24.382 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:11:24.384 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:11:24.384 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:24.384 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:11:24.384 INFO:teuthology.orchestra.run.trial080.stdout:Reinstalling: 2026-02-22T23:11:24.384 INFO:teuthology.orchestra.run.trial080.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:24.385 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:11:24.385 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:11:24.385 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:11:24.385 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:11:24.385 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 803 k 2026-02-22T23:11:24.385 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 0 2026-02-22T23:11:24.386 INFO:teuthology.orchestra.run.trial080.stderr:Operation aborted. 2026-02-22T23:11:24.405 INFO:teuthology.orchestra.run.trial080.stdout:Is this ok [y/N]: 2026-02-22T23:11:24.405 DEBUG:teuthology.orchestra.run.trial080:> sudo yum reinstall -y kernel || true 2026-02-22T23:11:24.709 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:02 ago on Sun 22 Feb 2026 11:11:22 PM UTC. 2026-02-22T23:11:24.790 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:11:24.790 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:11:24.790 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-22T23:11:24.791 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:11:24.791 INFO:teuthology.orchestra.run.trial080.stdout:Reinstalling: 2026-02-22T23:11:24.791 INFO:teuthology.orchestra.run.trial080.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T23:11:24.791 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:11:24.791 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:11:24.791 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:11:24.791 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:11:24.792 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 803 k 2026-02-22T23:11:24.792 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 0 2026-02-22T23:11:24.792 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:11:25.526 INFO:teuthology.orchestra.run.trial080.stdout:kernel-5.14.0-681.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2026-02-22T23:11:25.526 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:11:25.526 INFO:teuthology.orchestra.run.trial080.stdout:Total 1.1 MB/s | 803 kB 00:00 2026-02-22T23:11:25.537 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:11:25.570 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:11:25.570 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:11:25.575 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:11:25.575 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:11:25.598 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:11:25.601 INFO:teuthology.orchestra.run.trial080.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:25.602 INFO:teuthology.orchestra.run.trial080.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:25.710 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:25.710 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T23:11:25.766 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T23:11:25.766 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:11:25.766 INFO:teuthology.orchestra.run.trial080.stdout:Reinstalled: 2026-02-22T23:11:25.766 INFO:teuthology.orchestra.run.trial080.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:25.766 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:11:25.766 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:11:25.793 DEBUG:teuthology.orchestra.run.trial080:> rpm -q kernel | sort -rV | head -n 1 2026-02-22T23:11:25.858 INFO:teuthology.orchestra.run.trial080.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-22T23:11:25.859 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-22T23:11:25.859 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T23:11:25.859 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2026-02-22T23:11:25.859 DEBUG:teuthology.orchestra.run.trial080:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T23:11:25.922 INFO:teuthology.orchestra.run.trial080.stdout:ttyS1 2026-02-22T23:11:25.948 DEBUG:teuthology.parallel:result is None 2026-02-22T23:11:25.948 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-22T23:11:25.960 INFO:teuthology.task.internal:Creating test directory... 2026-02-22T23:11:25.961 DEBUG:teuthology.orchestra.run.trial006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T23:11:25.963 DEBUG:teuthology.orchestra.run.trial040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T23:11:25.966 DEBUG:teuthology.orchestra.run.trial080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T23:11:25.990 DEBUG:teuthology.orchestra.run.trial189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T23:11:26.009 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-22T23:11:26.022 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-22T23:11:26.036 INFO:teuthology.task.internal:Creating archive directory... 2026-02-22T23:11:26.036 DEBUG:teuthology.orchestra.run.trial006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T23:11:26.039 DEBUG:teuthology.orchestra.run.trial040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T23:11:26.042 DEBUG:teuthology.orchestra.run.trial080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T23:11:26.046 DEBUG:teuthology.orchestra.run.trial189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T23:11:26.073 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-22T23:11:26.084 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-22T23:11:26.085 DEBUG:teuthology.orchestra.run.trial006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T23:11:26.095 DEBUG:teuthology.orchestra.run.trial040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T23:11:26.099 DEBUG:teuthology.orchestra.run.trial080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T23:11:26.103 DEBUG:teuthology.orchestra.run.trial189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T23:11:26.119 INFO:teuthology.orchestra.run.trial006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.122 INFO:teuthology.orchestra.run.trial040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.126 INFO:teuthology.orchestra.run.trial080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.128 INFO:teuthology.orchestra.run.trial006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.130 INFO:teuthology.orchestra.run.trial040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.131 INFO:teuthology.orchestra.run.trial189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.135 INFO:teuthology.orchestra.run.trial080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.140 INFO:teuthology.orchestra.run.trial189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:11:26.141 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-22T23:11:26.153 INFO:teuthology.task.internal:Configuring sudo... 2026-02-22T23:11:26.153 DEBUG:teuthology.orchestra.run.trial006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T23:11:26.171 DEBUG:teuthology.orchestra.run.trial040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T23:11:26.173 DEBUG:teuthology.orchestra.run.trial080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T23:11:26.178 DEBUG:teuthology.orchestra.run.trial189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T23:11:26.206 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-22T23:11:26.226 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-22T23:11:26.226 DEBUG:teuthology.orchestra.run.trial006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T23:11:26.234 DEBUG:teuthology.orchestra.run.trial040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T23:11:26.237 DEBUG:teuthology.orchestra.run.trial080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T23:11:26.242 DEBUG:teuthology.orchestra.run.trial189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T23:11:26.261 DEBUG:teuthology.orchestra.run.trial006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:26.379 DEBUG:teuthology.orchestra.run.trial006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:26.402 DEBUG:teuthology.orchestra.run.trial006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:26.537 DEBUG:teuthology.orchestra.run.trial006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:26.561 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:11:26.561 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T23:11:26.625 DEBUG:teuthology.orchestra.run.trial040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:26.713 DEBUG:teuthology.orchestra.run.trial040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:26.738 DEBUG:teuthology.orchestra.run.trial040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:26.896 DEBUG:teuthology.orchestra.run.trial040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:26.921 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:11:26.921 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T23:11:26.985 DEBUG:teuthology.orchestra.run.trial080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:27.072 DEBUG:teuthology.orchestra.run.trial080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:27.098 DEBUG:teuthology.orchestra.run.trial080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:27.231 DEBUG:teuthology.orchestra.run.trial080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:27.256 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:11:27.256 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T23:11:27.322 DEBUG:teuthology.orchestra.run.trial189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:27.427 DEBUG:teuthology.orchestra.run.trial189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:11:27.451 DEBUG:teuthology.orchestra.run.trial189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:27.608 DEBUG:teuthology.orchestra.run.trial189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:11:27.633 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:11:27.633 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T23:11:27.696 DEBUG:teuthology.orchestra.run.trial006:> sudo service rsyslog restart 2026-02-22T23:11:27.698 DEBUG:teuthology.orchestra.run.trial040:> sudo service rsyslog restart 2026-02-22T23:11:27.701 DEBUG:teuthology.orchestra.run.trial080:> sudo service rsyslog restart 2026-02-22T23:11:27.702 DEBUG:teuthology.orchestra.run.trial189:> sudo service rsyslog restart 2026-02-22T23:11:27.724 INFO:teuthology.orchestra.run.trial006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:11:27.727 INFO:teuthology.orchestra.run.trial040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:11:27.729 INFO:teuthology.orchestra.run.trial080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:11:27.764 INFO:teuthology.orchestra.run.trial189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:11:28.194 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-22T23:11:28.206 INFO:teuthology.task.internal:Starting timer... 2026-02-22T23:11:28.206 INFO:teuthology.run_tasks:Running task pcp... 2026-02-22T23:11:28.227 INFO:teuthology.run_tasks:Running task selinux... 2026-02-22T23:11:28.245 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-22T23:11:28.245 DEBUG:teuthology.orchestra.run.trial006:> sudo service auditd rotate 2026-02-22T23:11:28.275 INFO:teuthology.orchestra.run.trial006.stdout:Rotating logs: 2026-02-22T23:11:28.276 DEBUG:teuthology.orchestra.run.trial040:> sudo service auditd rotate 2026-02-22T23:11:28.305 INFO:teuthology.orchestra.run.trial040.stdout:Rotating logs: 2026-02-22T23:11:28.306 DEBUG:teuthology.orchestra.run.trial080:> sudo service auditd rotate 2026-02-22T23:11:28.336 INFO:teuthology.orchestra.run.trial080.stdout:Rotating logs: 2026-02-22T23:11:28.338 DEBUG:teuthology.orchestra.run.trial189:> sudo service auditd rotate 2026-02-22T23:11:28.368 INFO:teuthology.orchestra.run.trial189.stdout:Rotating logs: 2026-02-22T23:11:28.370 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-22T23:11:28.370 DEBUG:teuthology.orchestra.run.trial006:> /usr/sbin/getenforce 2026-02-22T23:11:28.387 INFO:teuthology.orchestra.run.trial006.stdout:Permissive 2026-02-22T23:11:28.387 DEBUG:teuthology.orchestra.run.trial040:> /usr/sbin/getenforce 2026-02-22T23:11:28.404 INFO:teuthology.orchestra.run.trial040.stdout:Permissive 2026-02-22T23:11:28.405 DEBUG:teuthology.orchestra.run.trial080:> /usr/sbin/getenforce 2026-02-22T23:11:28.422 INFO:teuthology.orchestra.run.trial080.stdout:Permissive 2026-02-22T23:11:28.422 DEBUG:teuthology.orchestra.run.trial189:> /usr/sbin/getenforce 2026-02-22T23:11:28.437 INFO:teuthology.orchestra.run.trial189.stdout:Permissive 2026-02-22T23:11:28.437 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial006.front.sepia.ceph.com': 'permissive', 'ubuntu@trial040.front.sepia.ceph.com': 'permissive', 'ubuntu@trial080.front.sepia.ceph.com': 'permissive', 'ubuntu@trial189.front.sepia.ceph.com': 'permissive'} 2026-02-22T23:11:28.438 DEBUG:teuthology.orchestra.run.trial006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:11:28.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:11:28.462 DEBUG:teuthology.orchestra.run.trial040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:11:28.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:11:28.488 DEBUG:teuthology.orchestra.run.trial080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:11:28.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:11:28.513 DEBUG:teuthology.orchestra.run.trial189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:11:28.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:11:28.539 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-22T23:11:28.539 DEBUG:teuthology.orchestra.run.trial006:> sudo /usr/sbin/setenforce permissive 2026-02-22T23:11:28.564 DEBUG:teuthology.orchestra.run.trial040:> sudo /usr/sbin/setenforce permissive 2026-02-22T23:11:28.588 DEBUG:teuthology.orchestra.run.trial080:> sudo /usr/sbin/setenforce permissive 2026-02-22T23:11:28.612 DEBUG:teuthology.orchestra.run.trial189:> sudo /usr/sbin/setenforce permissive 2026-02-22T23:11:28.637 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-22T23:11:28.650 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-22T23:11:28.659 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-22T23:11:28.660 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial006.front.sepia.ceph.com,trial040.front.sepia.ceph.com,trial080.front.sepia.ceph.com,trial189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-22T23:14:13.099 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial006.front.sepia.ceph.com'), Remote(name='ubuntu@trial040.front.sepia.ceph.com'), Remote(name='ubuntu@trial080.front.sepia.ceph.com'), Remote(name='ubuntu@trial189.front.sepia.ceph.com')] 2026-02-22T23:14:13.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial006.front.sepia.ceph.com' 2026-02-22T23:14:13.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:14:13.213 DEBUG:teuthology.orchestra.run.trial006:> true 2026-02-22T23:14:13.287 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial006.front.sepia.ceph.com' 2026-02-22T23:14:13.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial040.front.sepia.ceph.com' 2026-02-22T23:14:13.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:14:13.381 DEBUG:teuthology.orchestra.run.trial040:> true 2026-02-22T23:14:13.462 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial040.front.sepia.ceph.com' 2026-02-22T23:14:13.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial080.front.sepia.ceph.com' 2026-02-22T23:14:13.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:14:13.558 DEBUG:teuthology.orchestra.run.trial080:> true 2026-02-22T23:14:13.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial080.front.sepia.ceph.com' 2026-02-22T23:14:13.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial189.front.sepia.ceph.com' 2026-02-22T23:14:13.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:14:13.714 DEBUG:teuthology.orchestra.run.trial189:> true 2026-02-22T23:14:13.793 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial189.front.sepia.ceph.com' 2026-02-22T23:14:13.794 INFO:teuthology.run_tasks:Running task clock... 2026-02-22T23:14:13.807 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-22T23:14:13.807 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T23:14:13.808 DEBUG:teuthology.orchestra.run.trial006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:14:13.810 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T23:14:13.810 DEBUG:teuthology.orchestra.run.trial040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:14:13.812 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T23:14:13.812 DEBUG:teuthology.orchestra.run.trial080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:14:13.814 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T23:14:13.814 DEBUG:teuthology.orchestra.run.trial189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:14:13.836 INFO:teuthology.orchestra.run.trial006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-22T23:14:13.838 INFO:teuthology.orchestra.run.trial040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-22T23:14:13.841 INFO:teuthology.orchestra.run.trial080.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-22T23:14:13.847 INFO:teuthology.orchestra.run.trial006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-22T23:14:13.850 INFO:teuthology.orchestra.run.trial040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-22T23:14:13.853 INFO:teuthology.orchestra.run.trial080.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-22T23:14:13.860 INFO:teuthology.orchestra.run.trial189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-22T23:14:13.868 INFO:teuthology.orchestra.run.trial006.stderr:sudo: ntpd: command not found 2026-02-22T23:14:13.870 INFO:teuthology.orchestra.run.trial040.stderr:sudo: ntpd: command not found 2026-02-22T23:14:13.872 INFO:teuthology.orchestra.run.trial189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-22T23:14:13.875 INFO:teuthology.orchestra.run.trial080.stderr:sudo: ntpd: command not found 2026-02-22T23:14:13.878 INFO:teuthology.orchestra.run.trial006.stdout:506 Cannot talk to daemon 2026-02-22T23:14:13.879 INFO:teuthology.orchestra.run.trial040.stdout:506 Cannot talk to daemon 2026-02-22T23:14:13.885 INFO:teuthology.orchestra.run.trial080.stdout:506 Cannot talk to daemon 2026-02-22T23:14:13.889 INFO:teuthology.orchestra.run.trial040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-22T23:14:13.890 INFO:teuthology.orchestra.run.trial006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-22T23:14:13.891 INFO:teuthology.orchestra.run.trial189.stderr:sudo: ntpd: command not found 2026-02-22T23:14:13.897 INFO:teuthology.orchestra.run.trial080.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-22T23:14:13.900 INFO:teuthology.orchestra.run.trial040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-22T23:14:13.901 INFO:teuthology.orchestra.run.trial189.stdout:506 Cannot talk to daemon 2026-02-22T23:14:13.902 INFO:teuthology.orchestra.run.trial006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-22T23:14:13.908 INFO:teuthology.orchestra.run.trial080.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-22T23:14:13.912 INFO:teuthology.orchestra.run.trial189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-22T23:14:13.924 INFO:teuthology.orchestra.run.trial189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-22T23:14:13.941 INFO:teuthology.orchestra.run.trial040.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:14:13.947 INFO:teuthology.orchestra.run.trial006.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:14:13.955 INFO:teuthology.orchestra.run.trial080.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:14:13.967 INFO:teuthology.orchestra.run.trial189.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:14:13.974 INFO:teuthology.orchestra.run.trial040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:14:13.974 INFO:teuthology.orchestra.run.trial040.stdout:=============================================================================== 2026-02-22T23:14:13.974 INFO:teuthology.orchestra.run.trial040.stdout:^? time.ecansol.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.974 INFO:teuthology.orchestra.run.trial040.stdout:^? 38.70.62.230 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.975 INFO:teuthology.orchestra.run.trial040.stdout:^? 172-234-25-10.ip.linodeu> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.975 INFO:teuthology.orchestra.run.trial040.stdout:^? 155.248.196.28 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.977 INFO:teuthology.orchestra.run.trial080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:14:13.977 INFO:teuthology.orchestra.run.trial080.stdout:=============================================================================== 2026-02-22T23:14:13.977 INFO:teuthology.orchestra.run.trial080.stdout:^? 99-28-14-242.lightspeed.> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.977 INFO:teuthology.orchestra.run.trial080.stdout:^? ns3.smatwebdesign.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.977 INFO:teuthology.orchestra.run.trial080.stdout:^? ip229.ip-51-81-226.us 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.977 INFO:teuthology.orchestra.run.trial080.stdout:^? us-east-2.clearnet.pw 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.985 INFO:teuthology.orchestra.run.trial189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:14:13.985 INFO:teuthology.orchestra.run.trial189.stdout:=============================================================================== 2026-02-22T23:14:13.986 INFO:teuthology.orchestra.run.trial189.stdout:^? ntp.sfo.icanbwell.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.986 INFO:teuthology.orchestra.run.trial189.stdout:^? 172-233-157-223.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.986 INFO:teuthology.orchestra.run.trial189.stdout:^? 139-177-202-26.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:13.986 INFO:teuthology.orchestra.run.trial189.stdout:^? gwallaceNTPpool.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:14.122 INFO:teuthology.orchestra.run.trial006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:14:14.122 INFO:teuthology.orchestra.run.trial006.stdout:=============================================================================== 2026-02-22T23:14:14.122 INFO:teuthology.orchestra.run.trial006.stdout:^? nc7j.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:14.122 INFO:teuthology.orchestra.run.trial006.stdout:^? 66.42.71.197.vultruserco> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:14.122 INFO:teuthology.orchestra.run.trial006.stdout:^? time.cloudflare.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:14.123 INFO:teuthology.orchestra.run.trial006.stdout:^? ntp6.kernfusion.at 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-22T23:14:14.125 INFO:teuthology.run_tasks:Running task install... 2026-02-22T23:14:14.139 DEBUG:teuthology.task.install:project ceph 2026-02-22T23:14:14.139 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}} 2026-02-22T23:14:14.139 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'} 2026-02-22T23:14:14.139 INFO:teuthology.task.install:Using flavor: default 2026-02-22T23:14:14.147 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2026-02-22T23:14:14.147 INFO:teuthology.task.install:extra packages: [] 2026-02-22T23:14:14.148 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-22T23:14:14.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:14:14.151 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-22T23:14:14.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:14:14.152 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-22T23:14:14.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:14:14.154 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-22T23:14:14.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:14:14.402 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-22T23:14:14.402 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-22T23:14:14.404 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-22T23:14:14.404 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-22T23:14:14.405 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-22T23:14:14.405 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-22T23:14:14.409 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-22T23:14:14.410 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-22T23:14:14.640 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-22T23:14:14.640 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:14:14.640 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-22T23:14:14.643 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-22T23:14:14.643 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:14:14.643 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-22T23:14:14.644 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-22T23:14:14.644 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:14:14.644 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-22T23:14:14.645 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-22T23:14:14.646 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:14:14.646 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-22T23:14:14.669 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-22T23:14:14.670 DEBUG:teuthology.orchestra.run.trial040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-22T23:14:14.670 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-22T23:14:14.670 DEBUG:teuthology.orchestra.run.trial006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-22T23:14:14.671 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-22T23:14:14.671 DEBUG:teuthology.orchestra.run.trial189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-22T23:14:14.672 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-22T23:14:14.672 DEBUG:teuthology.orchestra.run.trial080:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-22T23:14:14.734 DEBUG:teuthology.orchestra.run.trial040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-22T23:14:14.735 DEBUG:teuthology.orchestra.run.trial006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-22T23:14:14.736 DEBUG:teuthology.orchestra.run.trial080:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-22T23:14:14.738 DEBUG:teuthology.orchestra.run.trial189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-22T23:14:14.807 DEBUG:teuthology.orchestra.run.trial040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-22T23:14:14.809 DEBUG:teuthology.orchestra.run.trial006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-22T23:14:14.810 DEBUG:teuthology.orchestra.run.trial080:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-22T23:14:14.811 DEBUG:teuthology.orchestra.run.trial189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-22T23:14:14.871 INFO:teuthology.orchestra.run.trial040.stdout:check_obsoletes = 1 2026-02-22T23:14:14.873 DEBUG:teuthology.orchestra.run.trial040:> sudo yum clean all 2026-02-22T23:14:14.874 INFO:teuthology.orchestra.run.trial080.stdout:check_obsoletes = 1 2026-02-22T23:14:14.874 INFO:teuthology.orchestra.run.trial006.stdout:check_obsoletes = 1 2026-02-22T23:14:14.875 INFO:teuthology.orchestra.run.trial189.stdout:check_obsoletes = 1 2026-02-22T23:14:14.878 DEBUG:teuthology.orchestra.run.trial006:> sudo yum clean all 2026-02-22T23:14:14.878 DEBUG:teuthology.orchestra.run.trial080:> sudo yum clean all 2026-02-22T23:14:14.879 DEBUG:teuthology.orchestra.run.trial189:> sudo yum clean all 2026-02-22T23:14:15.028 INFO:teuthology.orchestra.run.trial040.stdout:41 files removed 2026-02-22T23:14:15.035 INFO:teuthology.orchestra.run.trial080.stdout:41 files removed 2026-02-22T23:14:15.036 INFO:teuthology.orchestra.run.trial006.stdout:41 files removed 2026-02-22T23:14:15.037 INFO:teuthology.orchestra.run.trial189.stdout:41 files removed 2026-02-22T23:14:15.043 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-radosgw 2026-02-22T23:14:15.050 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-radosgw 2026-02-22T23:14:15.051 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-radosgw 2026-02-22T23:14:15.053 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-radosgw 2026-02-22T23:14:15.317 INFO:teuthology.orchestra.run.trial040.stdout:ceph packages for x86_64 651 kB/s | 86 kB 00:00 2026-02-22T23:14:15.320 INFO:teuthology.orchestra.run.trial080.stdout:ceph packages for x86_64 677 kB/s | 86 kB 00:00 2026-02-22T23:14:15.323 INFO:teuthology.orchestra.run.trial006.stdout:ceph packages for x86_64 671 kB/s | 86 kB 00:00 2026-02-22T23:14:15.326 INFO:teuthology.orchestra.run.trial189.stdout:ceph packages for x86_64 659 kB/s | 86 kB 00:00 2026-02-22T23:14:15.451 INFO:teuthology.orchestra.run.trial040.stdout:ceph noarch packages 110 kB/s | 12 kB 00:00 2026-02-22T23:14:15.454 INFO:teuthology.orchestra.run.trial080.stdout:ceph noarch packages 111 kB/s | 12 kB 00:00 2026-02-22T23:14:15.459 INFO:teuthology.orchestra.run.trial189.stdout:ceph noarch packages 112 kB/s | 12 kB 00:00 2026-02-22T23:14:15.461 INFO:teuthology.orchestra.run.trial006.stdout:ceph noarch packages 110 kB/s | 12 kB 00:00 2026-02-22T23:14:15.577 INFO:teuthology.orchestra.run.trial040.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-22T23:14:15.581 INFO:teuthology.orchestra.run.trial080.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-22T23:14:15.586 INFO:teuthology.orchestra.run.trial189.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-22T23:14:15.588 INFO:teuthology.orchestra.run.trial006.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-22T23:14:16.345 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-22T23:14:16.970 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.9 MB 00:01 2026-02-22T23:14:20.016 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-22T23:14:23.820 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-22T23:14:23.850 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 7.8 MB 00:01 2026-02-22T23:14:23.948 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-22T23:14:24.737 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - Extras packages 131 kB/s | 20 kB 00:00 2026-02-22T23:14:27.731 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 27 MB 00:03 2026-02-22T23:14:29.696 INFO:teuthology.orchestra.run.trial080.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 20 MB 00:04 2026-02-22T23:14:32.873 INFO:teuthology.orchestra.run.trial080.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-22T23:14:34.086 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:14:34.087 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:14:34.087 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repository Size 2026-02-22T23:14:34.087 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:14:34.087 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:14:34.087 INFO:teuthology.orchestra.run.trial080.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:14:34.087 INFO:teuthology.orchestra.run.trial080.stdout:Upgrading: 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout:Installing dependencies: 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-22T23:14:34.088 INFO:teuthology.orchestra.run.trial080.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout:Installing weak dependencies: 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout:Install 33 Packages 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout:Upgrade 2 Packages 2026-02-22T23:14:34.089 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:34.090 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 63 M 2026-02-22T23:14:34.090 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:14:34.974 INFO:teuthology.orchestra.run.trial080.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 15 MB/s | 5.5 MB 00:00 2026-02-22T23:14:35.014 INFO:teuthology.orchestra.run.trial080.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 628 kB/s | 25 kB 00:00 2026-02-22T23:14:35.078 INFO:teuthology.orchestra.run.trial080.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 15 MB/s | 999 kB 00:00 2026-02-22T23:14:35.134 INFO:teuthology.orchestra.run.trial080.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 8.8 MB/s | 503 kB 00:00 2026-02-22T23:14:35.310 INFO:teuthology.orchestra.run.trial080.stdout:(5/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 15 MB/s | 11 MB 00:00 2026-02-22T23:14:35.351 INFO:teuthology.orchestra.run.trial080.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 1.1 MB/s | 45 kB 00:00 2026-02-22T23:14:35.391 INFO:teuthology.orchestra.run.trial080.stdout:(7/35): python3-ceph-common-19.2.3-689.gf90f4ea 3.4 MB/s | 142 kB 00:00 2026-02-22T23:14:35.433 INFO:teuthology.orchestra.run.trial080.stdout:(8/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 3.9 MB/s | 165 kB 00:00 2026-02-22T23:14:35.445 INFO:teuthology.orchestra.run.trial080.stdout:(9/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_64 17 MB/s | 5.4 MB 00:00 2026-02-22T23:14:35.480 INFO:teuthology.orchestra.run.trial080.stdout:(10/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 25 MB/s | 22 MB 00:00 2026-02-22T23:14:35.481 INFO:teuthology.orchestra.run.trial080.stdout:(11/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 8.2 MB/s | 303 kB 00:00 2026-02-22T23:14:35.482 INFO:teuthology.orchestra.run.trial080.stdout:(12/35): python3-rados-19.2.3-689.gf90f4eae.el9 6.5 MB/s | 322 kB 00:00 2026-02-22T23:14:35.492 INFO:teuthology.orchestra.run.trial080.stdout:(13/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 8.2 MB/s | 100 kB 00:00 2026-02-22T23:14:35.921 INFO:teuthology.orchestra.run.trial080.stdout:(14/35): mailcap-2.1.49-5.el9.noarch.rpm 75 kB/s | 33 kB 00:00 2026-02-22T23:14:36.127 INFO:teuthology.orchestra.run.trial080.stdout:(15/35): boost-program-options-1.75.0-13.el9.x8 504 kB/s | 104 kB 00:00 2026-02-22T23:14:36.159 INFO:teuthology.orchestra.run.trial080.stdout:(16/35): pciutils-3.7.0-7.el9.x86_64.rpm 140 kB/s | 93 kB 00:00 2026-02-22T23:14:36.368 INFO:teuthology.orchestra.run.trial080.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 681 kB/s | 164 kB 00:00 2026-02-22T23:14:36.452 INFO:teuthology.orchestra.run.trial080.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 540 kB/s | 45 kB 00:00 2026-02-22T23:14:36.708 INFO:teuthology.orchestra.run.trial080.stdout:(19/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 292 kB/s | 160 kB 00:00 2026-02-22T23:14:37.241 INFO:teuthology.orchestra.run.trial080.stdout:(20/35): librdkafka-1.6.1-102.el9.x86_64.rpm 840 kB/s | 662 kB 00:00 2026-02-22T23:14:37.260 INFO:teuthology.orchestra.run.trial080.stdout:(21/35): nvme-cli-2.16-1.el9.x86_64.rpm 663 kB/s | 1.2 MB 00:01 2026-02-22T23:14:37.343 INFO:teuthology.orchestra.run.trial080.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 445 kB/s | 37 kB 00:00 2026-02-22T23:14:37.413 INFO:teuthology.orchestra.run.trial080.stdout:(23/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 415 kB/s | 292 kB 00:00 2026-02-22T23:14:37.427 INFO:teuthology.orchestra.run.trial080.stdout:(24/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 793 kB/s | 66 kB 00:00 2026-02-22T23:14:37.560 INFO:teuthology.orchestra.run.trial080.stdout:(25/35): qatlib-25.08.0-2.el9.x86_64.rpm 751 kB/s | 240 kB 00:00 2026-02-22T23:14:37.779 INFO:teuthology.orchestra.run.trial080.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 113 kB/s | 25 kB 00:00 2026-02-22T23:14:37.881 INFO:teuthology.orchestra.run.trial080.stdout:(27/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 657 kB/s | 308 kB 00:00 2026-02-22T23:14:37.926 INFO:teuthology.orchestra.run.trial080.stdout:(28/35): liboath-2.6.12-1.el9.x86_64.rpm 332 kB/s | 49 kB 00:00 2026-02-22T23:14:37.959 INFO:teuthology.orchestra.run.trial080.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 860 kB/s | 67 kB 00:00 2026-02-22T23:14:38.058 INFO:teuthology.orchestra.run.trial080.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2026-02-22T23:14:38.137 INFO:teuthology.orchestra.run.trial080.stdout:(31/35): libarrow-9.0.0-15.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2026-02-22T23:14:38.254 INFO:teuthology.orchestra.run.trial080.stdout:(32/35): librados2-19.2.3-689.gf90f4eae.el9.x86 30 MB/s | 3.4 MB 00:00 2026-02-22T23:14:38.255 INFO:teuthology.orchestra.run.trial080.stdout:(33/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2026-02-22T23:14:38.309 INFO:teuthology.orchestra.run.trial080.stdout:(34/35): thrift-0.15.0-4.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2026-02-22T23:14:38.313 INFO:teuthology.orchestra.run.trial080.stdout:(35/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 53 MB/s | 3.2 MB 00:00 2026-02-22T23:14:38.316 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:14:38.316 INFO:teuthology.orchestra.run.trial080.stdout:Total 15 MB/s | 63 MB 00:04 2026-02-22T23:14:38.410 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:14:38.430 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:14:38.431 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:14:38.563 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - CRB 942 kB/s | 7.8 MB 00:08 2026-02-22T23:14:38.685 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:14:38.686 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:14:38.982 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:14:38.991 INFO:teuthology.orchestra.run.trial080.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-22T23:14:38.994 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-22T23:14:39.006 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-22T23:14:39.155 INFO:teuthology.orchestra.run.trial080.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-22T23:14:39.159 INFO:teuthology.orchestra.run.trial080.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:39.189 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:39.232 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:14:39.233 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:39.245 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:39.249 INFO:teuthology.orchestra.run.trial080.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-22T23:14:39.258 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-22T23:14:39.280 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:14:39.281 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:39.295 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:39.329 INFO:teuthology.orchestra.run.trial080.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-22T23:14:39.333 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-22T23:14:39.339 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-22T23:14:39.341 INFO:teuthology.orchestra.run.trial080.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-22T23:14:39.344 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-22T23:14:39.350 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-22T23:14:39.489 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:14:39.490 INFO:teuthology.orchestra.run.trial080.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:14:39.517 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:14:39.520 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-22T23:14:39.527 INFO:teuthology.orchestra.run.trial080.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-22T23:14:39.753 INFO:teuthology.orchestra.run.trial080.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-22T23:14:39.754 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:14:39.771 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:14:39.775 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-22T23:14:39.781 INFO:teuthology.orchestra.run.trial080.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-22T23:14:39.784 INFO:teuthology.orchestra.run.trial080.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-22T23:14:39.785 INFO:teuthology.orchestra.run.trial080.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:14:39.803 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:14:39.922 INFO:teuthology.orchestra.run.trial080.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-22T23:14:39.924 INFO:teuthology.orchestra.run.trial080.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:14:40.099 INFO:teuthology.orchestra.run.trial189.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 20 kB 00:00 2026-02-22T23:14:40.204 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:14:40.204 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T23:14:40.204 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:40.239 INFO:teuthology.orchestra.run.trial080.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-22T23:14:40.241 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-22T23:14:40.964 INFO:teuthology.orchestra.run.trial189.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2026-02-22T23:14:41.220 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:41.222 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:41.505 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:41.508 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:14:41.548 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:14:41.548 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-22T23:14:41.548 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-22T23:14:41.548 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:41.552 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:14:44.134 INFO:teuthology.orchestra.run.trial189.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-22T23:14:44.466 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - AppStream 995 kB/s | 27 MB 00:27 2026-02-22T23:14:45.328 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:14:45.329 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:14:45.329 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repository Size 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout:Upgrading: 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-22T23:14:45.330 INFO:teuthology.orchestra.run.trial189.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-22T23:14:45.331 INFO:teuthology.orchestra.run.trial189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-22T23:14:45.332 INFO:teuthology.orchestra.run.trial189.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout:Installing weak dependencies: 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout:Install 33 Packages 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout:Upgrade 2 Packages 2026-02-22T23:14:45.333 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:45.334 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 63 M 2026-02-22T23:14:45.334 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:14:46.036 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /sys 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /proc 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /mnt 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /var/tmp 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /home 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /root 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /tmp 2026-02-22T23:14:46.037 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:46.059 INFO:teuthology.orchestra.run.trial189.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 19 MB/s | 5.5 MB 00:00 2026-02-22T23:14:46.091 INFO:teuthology.orchestra.run.trial189.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 778 kB/s | 25 kB 00:00 2026-02-22T23:14:46.149 INFO:teuthology.orchestra.run.trial189.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 17 MB/s | 999 kB 00:00 2026-02-22T23:14:46.197 INFO:teuthology.orchestra.run.trial189.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 10 MB/s | 503 kB 00:00 2026-02-22T23:14:46.470 INFO:teuthology.orchestra.run.trial189.stdout:(5/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_64 20 MB/s | 5.4 MB 00:00 2026-02-22T23:14:46.489 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:14:46.495 INFO:teuthology.orchestra.run.trial189.stdout:(6/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 15 MB/s | 11 MB 00:00 2026-02-22T23:14:46.499 INFO:teuthology.orchestra.run.trial189.stdout:(7/35): python3-ceph-argparse-19.2.3-689.gf90f4 1.5 MB/s | 45 kB 00:00 2026-02-22T23:14:46.512 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:14:46.513 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:14:46.513 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-22T23:14:46.513 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:14:46.513 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:14:46.513 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:46.513 INFO:teuthology.orchestra.run.trial189.stdout:(8/35): python3-ceph-common-19.2.3-689.gf90f4ea 7.9 MB/s | 142 kB 00:00 2026-02-22T23:14:46.514 INFO:teuthology.orchestra.run.trial080.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:14:46.523 INFO:teuthology.orchestra.run.trial189.stdout:(9/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 6.7 MB/s | 165 kB 00:00 2026-02-22T23:14:46.527 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:14:46.527 INFO:teuthology.orchestra.run.trial080.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:14:46.541 INFO:teuthology.orchestra.run.trial189.stdout:(10/35): python3-rados-19.2.3-689.gf90f4eae.el9 11 MB/s | 322 kB 00:00 2026-02-22T23:14:46.554 INFO:teuthology.orchestra.run.trial189.stdout:(11/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 9.7 MB/s | 303 kB 00:00 2026-02-22T23:14:46.568 INFO:teuthology.orchestra.run.trial189.stdout:(12/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 3.6 MB/s | 100 kB 00:00 2026-02-22T23:14:46.584 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:14:46.584 INFO:teuthology.orchestra.run.trial080.stdout:Creating group 'qat' with GID 992. 2026-02-22T23:14:46.584 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:46.624 INFO:teuthology.orchestra.run.trial189.stdout:(13/35): mailcap-2.1.49-5.el9.noarch.rpm 479 kB/s | 33 kB 00:00 2026-02-22T23:14:46.672 INFO:teuthology.orchestra.run.trial189.stdout:(14/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 24 MB/s | 22 MB 00:00 2026-02-22T23:14:46.693 INFO:teuthology.orchestra.run.trial189.stdout:(15/35): pciutils-3.7.0-7.el9.x86_64.rpm 1.3 MB/s | 93 kB 00:00 2026-02-22T23:14:46.696 INFO:teuthology.orchestra.run.trial189.stdout:(16/35): nvme-cli-2.16-1.el9.x86_64.rpm 9.0 MB/s | 1.2 MB 00:00 2026-02-22T23:14:46.860 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-22T23:14:46.860 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-22T23:14:46.860 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-22T23:14:46.860 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-22T23:14:46.860 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:46.860 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-22T23:14:46.861 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-22T23:14:46.862 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-22T23:14:46.863 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-22T23:14:46.919 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout:Upgraded: 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.920 INFO:teuthology.orchestra.run.trial080.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-22T23:14:46.921 INFO:teuthology.orchestra.run.trial080.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-22T23:14:46.922 INFO:teuthology.orchestra.run.trial080.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-22T23:14:46.923 INFO:teuthology.orchestra.run.trial080.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-22T23:14:46.923 INFO:teuthology.orchestra.run.trial080.stdout: re2-1:20211101-20.el9.x86_64 2026-02-22T23:14:46.923 INFO:teuthology.orchestra.run.trial080.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-22T23:14:46.923 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:46.923 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:14:46.962 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-test 2026-02-22T23:14:47.273 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:15 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:14:47.357 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:14:47.358 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:14:47.358 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repository Size 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout:Installing dependencies: 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:47.359 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:14:47.360 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:14:47.360 INFO:teuthology.orchestra.run.trial080.stdout:Install 5 Packages 2026-02-22T23:14:47.360 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:47.360 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 50 M 2026-02-22T23:14:47.360 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 212 M 2026-02-22T23:14:47.360 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:14:47.387 INFO:teuthology.orchestra.run.trial189.stdout:(17/35): boost-program-options-1.75.0-13.el9.x8 145 kB/s | 104 kB 00:00 2026-02-22T23:14:47.416 INFO:teuthology.orchestra.run.trial189.stdout:(18/35): libnbd-1.20.3-4.el9.x86_64.rpm 227 kB/s | 164 kB 00:00 2026-02-22T23:14:47.452 INFO:teuthology.orchestra.run.trial189.stdout:(19/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 212 kB/s | 160 kB 00:00 2026-02-22T23:14:47.481 INFO:teuthology.orchestra.run.trial080.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 3.9 MB/s | 233 kB 00:00 2026-02-22T23:14:47.516 INFO:teuthology.orchestra.run.trial080.stdout:(2/5): socat-1.7.4.1-8.el9.x86_64.rpm 8.4 MB/s | 303 kB 00:00 2026-02-22T23:14:47.547 INFO:teuthology.orchestra.run.trial189.stdout:(20/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 283 kB/s | 45 kB 00:00 2026-02-22T23:14:47.549 INFO:teuthology.orchestra.run.trial080.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2026-02-22T23:14:47.625 INFO:teuthology.orchestra.run.trial080.stdout:(4/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 807 kB/s | 163 kB 00:00 2026-02-22T23:14:47.731 INFO:teuthology.orchestra.run.trial189.stdout:(21/35): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2026-02-22T23:14:47.769 INFO:teuthology.orchestra.run.trial189.stdout:(22/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 923 kB/s | 292 kB 00:00 2026-02-22T23:14:47.889 INFO:teuthology.orchestra.run.trial189.stdout:(23/35): qatlib-service-25.08.0-2.el9.x86_64.rp 235 kB/s | 37 kB 00:00 2026-02-22T23:14:47.926 INFO:teuthology.orchestra.run.trial189.stdout:(24/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 420 kB/s | 66 kB 00:00 2026-02-22T23:14:47.943 INFO:teuthology.orchestra.run.trial189.stdout:(25/35): qatlib-25.08.0-2.el9.x86_64.rpm 605 kB/s | 240 kB 00:00 2026-02-22T23:14:47.958 INFO:teuthology.orchestra.run.trial189.stdout:(26/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.4 MB/s | 308 kB 00:00 2026-02-22T23:14:47.965 INFO:teuthology.orchestra.run.trial189.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 7.0 MB/s | 49 kB 00:00 2026-02-22T23:14:47.967 INFO:teuthology.orchestra.run.trial189.stdout:(28/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.0 MB/s | 25 kB 00:00 2026-02-22T23:14:47.975 INFO:teuthology.orchestra.run.trial189.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 6.5 MB/s | 67 kB 00:00 2026-02-22T23:14:47.981 INFO:teuthology.orchestra.run.trial189.stdout:(30/35): libarrow-9.0.0-15.el9.x86_64.rpm 81 MB/s | 4.4 MB 00:00 2026-02-22T23:14:47.987 INFO:teuthology.orchestra.run.trial189.stdout:(31/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 41 MB/s | 838 kB 00:00 2026-02-22T23:14:47.994 INFO:teuthology.orchestra.run.trial189.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 129 MB/s | 1.6 MB 00:00 2026-02-22T23:14:47.997 INFO:teuthology.orchestra.run.trial189.stdout:(33/35): re2-20211101-20.el9.x86_64.rpm 8.5 MB/s | 191 kB 00:00 2026-02-22T23:14:48.441 INFO:teuthology.orchestra.run.trial189.stdout:(34/35): librados2-19.2.3-689.gf90f4eae.el9.x86 7.6 MB/s | 3.4 MB 00:00 2026-02-22T23:14:48.613 INFO:teuthology.orchestra.run.trial080.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 42 MB/s | 50 MB 00:01 2026-02-22T23:14:48.615 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:14:48.615 INFO:teuthology.orchestra.run.trial080.stdout:Total 40 MB/s | 50 MB 00:01 2026-02-22T23:14:48.643 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:14:48.653 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:14:48.653 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:14:48.740 INFO:teuthology.orchestra.run.trial189.stdout:(35/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 4.2 MB/s | 3.2 MB 00:00 2026-02-22T23:14:48.746 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:14:48.746 INFO:teuthology.orchestra.run.trial189.stdout:Total 18 MB/s | 63 MB 00:03 2026-02-22T23:14:48.747 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 7.8 MB 00:01 2026-02-22T23:14:48.838 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:14:48.839 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:14:48.840 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:14:48.857 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:14:48.857 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:14:48.870 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 27 MB 00:24 2026-02-22T23:14:49.051 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:14:49.058 INFO:teuthology.orchestra.run.trial080.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-22T23:14:49.062 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-22T23:14:49.070 INFO:teuthology.orchestra.run.trial080.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-22T23:14:49.071 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:14:49.113 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:14:49.113 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:14:49.407 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:14:49.416 INFO:teuthology.orchestra.run.trial189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-22T23:14:49.418 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-22T23:14:49.430 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-22T23:14:49.585 INFO:teuthology.orchestra.run.trial189.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-22T23:14:49.588 INFO:teuthology.orchestra.run.trial189.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:49.619 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:49.664 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:14:49.665 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:49.677 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:49.682 INFO:teuthology.orchestra.run.trial189.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-22T23:14:49.692 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-22T23:14:49.716 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:14:49.716 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:49.730 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:49.765 INFO:teuthology.orchestra.run.trial189.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-22T23:14:49.770 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-22T23:14:49.777 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-22T23:14:49.779 INFO:teuthology.orchestra.run.trial189.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-22T23:14:49.782 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-22T23:14:49.787 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-22T23:14:49.929 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:14:49.930 INFO:teuthology.orchestra.run.trial189.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:14:49.956 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:14:49.958 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-22T23:14:49.965 INFO:teuthology.orchestra.run.trial189.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-22T23:14:50.194 INFO:teuthology.orchestra.run.trial189.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-22T23:14:50.196 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:14:50.212 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:14:50.217 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-22T23:14:50.223 INFO:teuthology.orchestra.run.trial189.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-22T23:14:50.226 INFO:teuthology.orchestra.run.trial189.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-22T23:14:50.226 INFO:teuthology.orchestra.run.trial189.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:14:50.245 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:14:50.274 INFO:teuthology.orchestra.run.trial040.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 20 kB 00:00 2026-02-22T23:14:50.363 INFO:teuthology.orchestra.run.trial189.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-22T23:14:50.365 INFO:teuthology.orchestra.run.trial189.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:14:50.640 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:14:50.640 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T23:14:50.640 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:50.646 INFO:teuthology.orchestra.run.trial040.stdout:Extra Packages for Enterprise Linux 65 MB/s | 20 MB 00:00 2026-02-22T23:14:50.675 INFO:teuthology.orchestra.run.trial189.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-22T23:14:50.677 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-22T23:14:51.321 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:14:51.339 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:14:51.565 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:14:51.566 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-22T23:14:51.566 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-22T23:14:51.566 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-22T23:14:51.566 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-22T23:14:51.620 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:51.621 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:14:51.654 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph 2026-02-22T23:14:51.670 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:51.673 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:51.963 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:19 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:14:51.968 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:51.972 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:14:52.014 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:14:52.015 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-22T23:14:52.015 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-22T23:14:52.015 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:52.018 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:14:52.050 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:14:52.053 INFO:teuthology.orchestra.run.trial080.stdout:====================================================================================== 2026-02-22T23:14:52.053 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repository Size 2026-02-22T23:14:52.053 INFO:teuthology.orchestra.run.trial080.stdout:====================================================================================== 2026-02-22T23:14:52.053 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:14:52.053 INFO:teuthology.orchestra.run.trial080.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-22T23:14:52.053 INFO:teuthology.orchestra.run.trial080.stdout:Installing dependencies: 2026-02-22T23:14:52.053 INFO:teuthology.orchestra.run.trial080.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-22T23:14:52.054 INFO:teuthology.orchestra.run.trial080.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-22T23:14:52.055 INFO:teuthology.orchestra.run.trial080.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-22T23:14:52.056 INFO:teuthology.orchestra.run.trial080.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-22T23:14:52.057 INFO:teuthology.orchestra.run.trial080.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-22T23:14:52.058 INFO:teuthology.orchestra.run.trial080.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-22T23:14:52.059 INFO:teuthology.orchestra.run.trial080.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-22T23:14:52.060 INFO:teuthology.orchestra.run.trial080.stdout:Installing weak dependencies: 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout:====================================================================================== 2026-02-22T23:14:52.061 INFO:teuthology.orchestra.run.trial080.stdout:Install 99 Packages 2026-02-22T23:14:52.062 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:14:52.062 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 95 M 2026-02-22T23:14:52.063 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 520 M 2026-02-22T23:14:52.063 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:14:52.779 INFO:teuthology.orchestra.run.trial080.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 34 kB/s | 6.5 kB 00:00 2026-02-22T23:14:53.098 INFO:teuthology.orchestra.run.trial080.stdout:(2/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 2.1 MB/s | 1.1 MB 00:00 2026-02-22T23:14:53.153 INFO:teuthology.orchestra.run.trial080.stdout:(3/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 4.3 MB/s | 2.4 MB 00:00 2026-02-22T23:14:53.164 INFO:teuthology.orchestra.run.trial080.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 2.6 MB/s | 31 kB 00:00 2026-02-22T23:14:53.177 INFO:teuthology.orchestra.run.trial080.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 12 MB/s | 150 kB 00:00 2026-02-22T23:14:53.342 INFO:teuthology.orchestra.run.trial080.stdout:(6/99): ceph-mgr-dashboard-19.2.3-689.gf90f4eae 23 MB/s | 3.8 MB 00:00 2026-02-22T23:14:53.575 INFO:teuthology.orchestra.run.trial080.stdout:(7/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 6.0 MB/s | 4.7 MB 00:00 2026-02-22T23:14:53.607 INFO:teuthology.orchestra.run.trial080.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 665 kB/s | 22 kB 00:00 2026-02-22T23:14:53.644 INFO:teuthology.orchestra.run.trial080.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 6.9 MB/s | 253 kB 00:00 2026-02-22T23:14:53.668 INFO:teuthology.orchestra.run.trial080.stdout:(10/99): ceph-mgr-diskprediction-local-19.2.3-6 23 MB/s | 7.4 MB 00:00 2026-02-22T23:14:53.671 INFO:teuthology.orchestra.run.trial080.stdout:(11/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 1.7 MB/s | 49 kB 00:00 2026-02-22T23:14:53.687 INFO:teuthology.orchestra.run.trial080.stdout:(12/99): ceph-prometheus-alerts-19.2.3-689.gf90 868 kB/s | 17 kB 00:00 2026-02-22T23:14:53.713 INFO:teuthology.orchestra.run.trial080.stdout:(13/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_ 28 MB/s | 17 MB 00:00 2026-02-22T23:14:53.714 INFO:teuthology.orchestra.run.trial080.stdout:(14/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 6.8 MB/s | 299 kB 00:00 2026-02-22T23:14:53.716 INFO:teuthology.orchestra.run.trial080.stdout:(15/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 26 MB/s | 769 kB 00:00 2026-02-22T23:14:53.825 INFO:teuthology.orchestra.run.trial040.stdout:lab-extras 1.8 MB/s | 50 kB 00:00 2026-02-22T23:14:54.119 INFO:teuthology.orchestra.run.trial080.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 100 kB/s | 40 kB 00:00 2026-02-22T23:14:54.197 INFO:teuthology.orchestra.run.trial080.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 150 kB/s | 72 kB 00:00 2026-02-22T23:14:54.437 INFO:teuthology.orchestra.run.trial080.stdout:(18/99): libquadmath-11.5.0-14.el9.x86_64.rpm 771 kB/s | 184 kB 00:00 2026-02-22T23:14:54.525 INFO:teuthology.orchestra.run.trial080.stdout:(19/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 432 kB/s | 351 kB 00:00 2026-02-22T23:14:54.753 INFO:teuthology.orchestra.run.trial080.stdout:(20/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 800 kB/s | 253 kB 00:00 2026-02-22T23:14:54.915 INFO:teuthology.orchestra.run.trial080.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 661 kB/s | 106 kB 00:00 2026-02-22T23:14:55.028 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:14:55.029 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:14:55.029 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repository Size 2026-02-22T23:14:55.029 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:14:55.029 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:14:55.029 INFO:teuthology.orchestra.run.trial040.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:14:55.029 INFO:teuthology.orchestra.run.trial040.stdout:Upgrading: 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-22T23:14:55.030 INFO:teuthology.orchestra.run.trial040.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-22T23:14:55.031 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout:Installing weak dependencies: 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-22T23:14:55.032 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:14:55.033 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:14:55.033 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:14:55.033 INFO:teuthology.orchestra.run.trial040.stdout:Install 33 Packages 2026-02-22T23:14:55.033 INFO:teuthology.orchestra.run.trial040.stdout:Upgrade 2 Packages 2026-02-22T23:14:55.033 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:14:55.034 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 63 M 2026-02-22T23:14:55.034 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:14:55.076 INFO:teuthology.orchestra.run.trial080.stdout:(22/99): python3-pycparser-2.20-6.el9.noarch.rp 837 kB/s | 135 kB 00:00 2026-02-22T23:14:55.315 INFO:teuthology.orchestra.run.trial080.stdout:(23/99): python3-pyparsing-2.4.7-9.el9.noarch.r 630 kB/s | 150 kB 00:00 2026-02-22T23:14:55.476 INFO:teuthology.orchestra.run.trial080.stdout:(24/99): python3-requests-2.25.1-10.el9.noarch. 784 kB/s | 126 kB 00:00 2026-02-22T23:14:55.793 INFO:teuthology.orchestra.run.trial080.stdout:(25/99): python3-urllib3-1.26.5-7.el9.noarch.rp 689 kB/s | 218 kB 00:00 2026-02-22T23:14:55.839 INFO:teuthology.orchestra.run.trial040.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 19 MB/s | 5.5 MB 00:00 2026-02-22T23:14:55.870 INFO:teuthology.orchestra.run.trial040.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 819 kB/s | 25 kB 00:00 2026-02-22T23:14:55.877 INFO:teuthology.orchestra.run.trial080.stdout:(26/99): python3-cryptography-36.0.1-5.el9.x86_ 943 kB/s | 1.2 MB 00:01 2026-02-22T23:14:55.925 INFO:teuthology.orchestra.run.trial040.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 18 MB/s | 999 kB 00:00 2026-02-22T23:14:55.963 INFO:teuthology.orchestra.run.trial080.stdout:(27/99): zip-3.0-35.el9.x86_64.rpm 3.0 MB/s | 266 kB 00:00 2026-02-22T23:14:55.973 INFO:teuthology.orchestra.run.trial040.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 10 MB/s | 503 kB 00:00 2026-02-22T23:14:56.031 INFO:teuthology.orchestra.run.trial080.stdout:(28/99): unzip-6.0-59.el9.x86_64.rpm 763 kB/s | 182 kB 00:00 2026-02-22T23:14:56.066 INFO:teuthology.orchestra.run.trial080.stdout:(29/99): libgfortran-11.5.0-14.el9.x86_64.rpm 408 kB/s | 794 kB 00:01 2026-02-22T23:14:56.150 INFO:teuthology.orchestra.run.trial040.stdout:(5/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 18 MB/s | 11 MB 00:00 2026-02-22T23:14:56.194 INFO:teuthology.orchestra.run.trial040.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 1.0 MB/s | 45 kB 00:00 2026-02-22T23:14:56.216 INFO:teuthology.orchestra.run.trial040.stdout:(7/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_64 22 MB/s | 5.4 MB 00:00 2026-02-22T23:14:56.221 INFO:teuthology.orchestra.run.trial080.stdout:(30/99): flexiblas-3.0.4-9.el9.x86_64.rpm 114 kB/s | 30 kB 00:00 2026-02-22T23:14:56.228 INFO:teuthology.orchestra.run.trial040.stdout:(8/35): python3-ceph-common-19.2.3-689.gf90f4ea 4.1 MB/s | 142 kB 00:00 2026-02-22T23:14:56.243 INFO:teuthology.orchestra.run.trial040.stdout:(9/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 5.8 MB/s | 165 kB 00:00 2026-02-22T23:14:56.261 INFO:teuthology.orchestra.run.trial040.stdout:(10/35): python3-rados-19.2.3-689.gf90f4eae.el9 9.8 MB/s | 322 kB 00:00 2026-02-22T23:14:56.278 INFO:teuthology.orchestra.run.trial040.stdout:(11/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 8.5 MB/s | 303 kB 00:00 2026-02-22T23:14:56.292 INFO:teuthology.orchestra.run.trial040.stdout:(12/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 3.1 MB/s | 100 kB 00:00 2026-02-22T23:14:56.297 INFO:teuthology.orchestra.run.trial080.stdout:(31/99): flexiblas-openblas-openmp-3.0.4-9.el9. 64 kB/s | 15 kB 00:00 2026-02-22T23:14:56.321 INFO:teuthology.orchestra.run.trial040.stdout:(13/35): mailcap-2.1.49-5.el9.noarch.rpm 781 kB/s | 33 kB 00:00 2026-02-22T23:14:56.336 INFO:teuthology.orchestra.run.trial040.stdout:(14/35): nvme-cli-2.16-1.el9.x86_64.rpm 26 MB/s | 1.2 MB 00:00 2026-02-22T23:14:56.347 INFO:teuthology.orchestra.run.trial040.stdout:(15/35): pciutils-3.7.0-7.el9.x86_64.rpm 3.5 MB/s | 93 kB 00:00 2026-02-22T23:14:56.363 INFO:teuthology.orchestra.run.trial040.stdout:(16/35): boost-program-options-1.75.0-13.el9.x8 3.8 MB/s | 104 kB 00:00 2026-02-22T23:14:56.365 INFO:teuthology.orchestra.run.trial040.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 9.2 MB/s | 164 kB 00:00 2026-02-22T23:14:56.369 INFO:teuthology.orchestra.run.trial040.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 9.8 MB/s | 45 kB 00:00 2026-02-22T23:14:56.372 INFO:teuthology.orchestra.run.trial040.stdout:(19/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 18 MB/s | 160 kB 00:00 2026-02-22T23:14:56.382 INFO:teuthology.orchestra.run.trial040.stdout:(20/35): librdkafka-1.6.1-102.el9.x86_64.rpm 50 MB/s | 662 kB 00:00 2026-02-22T23:14:56.383 INFO:teuthology.orchestra.run.trial040.stdout:(21/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 26 MB/s | 292 kB 00:00 2026-02-22T23:14:56.388 INFO:teuthology.orchestra.run.trial040.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 8.2 MB/s | 37 kB 00:00 2026-02-22T23:14:56.389 INFO:teuthology.orchestra.run.trial040.stdout:(23/35): qatlib-25.08.0-2.el9.x86_64.rpm 37 MB/s | 240 kB 00:00 2026-02-22T23:14:56.392 INFO:teuthology.orchestra.run.trial040.stdout:(24/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 14 MB/s | 66 kB 00:00 2026-02-22T23:14:56.443 INFO:teuthology.orchestra.run.trial040.stdout:(25/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 24 MB/s | 22 MB 00:00 2026-02-22T23:14:56.453 INFO:teuthology.orchestra.run.trial040.stdout:(26/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.7 MB/s | 308 kB 00:00 2026-02-22T23:14:56.453 INFO:teuthology.orchestra.run.trial040.stdout:(27/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 2.4 MB/s | 25 kB 00:00 2026-02-22T23:14:56.457 INFO:teuthology.orchestra.run.trial040.stdout:(28/35): liboath-2.6.12-1.el9.x86_64.rpm 13 MB/s | 49 kB 00:00 2026-02-22T23:14:56.458 INFO:teuthology.orchestra.run.trial040.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 16 MB/s | 67 kB 00:00 2026-02-22T23:14:56.466 INFO:teuthology.orchestra.run.trial040.stdout:(30/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 94 MB/s | 838 kB 00:00 2026-02-22T23:14:56.468 INFO:teuthology.orchestra.run.trial040.stdout:(31/35): re2-20211101-20.el9.x86_64.rpm 19 MB/s | 191 kB 00:00 2026-02-22T23:14:56.473 INFO:teuthology.orchestra.run.trial040.stdout:(32/35): libarrow-9.0.0-15.el9.x86_64.rpm 57 MB/s | 4.4 MB 00:00 2026-02-22T23:14:56.478 INFO:teuthology.orchestra.run.trial040.stdout:(33/35): thrift-0.15.0-4.el9.x86_64.rpm 143 MB/s | 1.6 MB 00:00 2026-02-22T23:14:56.544 INFO:teuthology.orchestra.run.trial080.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 763 kB/s | 246 kB 00:00 2026-02-22T23:14:56.552 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:14:56.552 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /sys 2026-02-22T23:14:56.552 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /proc 2026-02-22T23:14:56.553 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /mnt 2026-02-22T23:14:56.553 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /var/tmp 2026-02-22T23:14:56.553 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /home 2026-02-22T23:14:56.553 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /root 2026-02-22T23:14:56.553 INFO:teuthology.orchestra.run.trial189.stdout:skipping the directory /tmp 2026-02-22T23:14:56.553 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:56.563 INFO:teuthology.orchestra.run.trial040.stdout:(34/35): librados2-19.2.3-689.gf90f4eae.el9.x86 36 MB/s | 3.4 MB 00:00 2026-02-22T23:14:56.587 INFO:teuthology.orchestra.run.trial040.stdout:(35/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 28 MB/s | 3.2 MB 00:00 2026-02-22T23:14:56.591 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:14:56.591 INFO:teuthology.orchestra.run.trial040.stdout:Total 40 MB/s | 63 MB 00:01 2026-02-22T23:14:56.621 INFO:teuthology.orchestra.run.trial080.stdout:(33/99): lua-5.4.4-4.el9.x86_64.rpm 583 kB/s | 188 kB 00:00 2026-02-22T23:14:56.630 INFO:teuthology.orchestra.run.trial080.stdout:(34/99): openblas-0.3.29-1.el9.x86_64.rpm 492 kB/s | 42 kB 00:00 2026-02-22T23:14:56.688 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:14:56.706 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:14:56.706 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:14:56.805 INFO:teuthology.orchestra.run.trial080.stdout:(35/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 3.9 MB/s | 3.0 MB 00:00 2026-02-22T23:14:56.889 INFO:teuthology.orchestra.run.trial080.stdout:(36/99): protobuf-3.14.0-17.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2026-02-22T23:14:56.962 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:14:56.962 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:14:56.978 INFO:teuthology.orchestra.run.trial080.stdout:(37/99): python3-devel-3.9.25-3.el9.x86_64.rpm 2.7 MB/s | 244 kB 00:00 2026-02-22T23:14:57.009 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:14:57.024 INFO:teuthology.orchestra.run.trial080.stdout:(38/99): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2026-02-22T23:14:57.033 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:14:57.033 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:14:57.033 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-22T23:14:57.033 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:14:57.033 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:14:57.033 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:57.035 INFO:teuthology.orchestra.run.trial189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:14:57.048 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:14:57.048 INFO:teuthology.orchestra.run.trial189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:14:57.067 INFO:teuthology.orchestra.run.trial080.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 2.7 MB/s | 249 kB 00:00 2026-02-22T23:14:57.095 INFO:teuthology.orchestra.run.trial080.stdout:(40/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 11 MB/s | 5.3 MB 00:00 2026-02-22T23:14:57.102 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:14:57.102 INFO:teuthology.orchestra.run.trial189.stdout:Creating group 'qat' with GID 992. 2026-02-22T23:14:57.102 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:57.105 INFO:teuthology.orchestra.run.trial080.stdout:(41/99): python3-libstoragemgmt-1.10.1-1.el9.x8 2.1 MB/s | 177 kB 00:00 2026-02-22T23:14:57.176 INFO:teuthology.orchestra.run.trial080.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 172 kB 00:00 2026-02-22T23:14:57.183 INFO:teuthology.orchestra.run.trial080.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 443 kB/s | 35 kB 00:00 2026-02-22T23:14:57.205 INFO:teuthology.orchestra.run.trial080.stdout:(44/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 8.9 MB/s | 1.2 MB 00:00 2026-02-22T23:14:57.256 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:14:57.265 INFO:teuthology.orchestra.run.trial080.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 5.3 MB/s | 442 kB 00:00 2026-02-22T23:14:57.265 INFO:teuthology.orchestra.run.trial040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-22T23:14:57.268 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-22T23:14:57.280 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-22T23:14:57.286 INFO:teuthology.orchestra.run.trial080.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 949 kB/s | 77 kB 00:00 2026-02-22T23:14:57.346 INFO:teuthology.orchestra.run.trial080.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 3.2 MB/s | 267 kB 00:00 2026-02-22T23:14:57.371 INFO:teuthology.orchestra.run.trial080.stdout:(48/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2026-02-22T23:14:57.379 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-22T23:14:57.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-22T23:14:57.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-22T23:14:57.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-22T23:14:57.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:57.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:14:57.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:57.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-22T23:14:57.381 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-22T23:14:57.382 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-22T23:14:57.383 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-22T23:14:57.383 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-22T23:14:57.383 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-22T23:14:57.383 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:14:57.383 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-22T23:14:57.383 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-22T23:14:57.431 INFO:teuthology.orchestra.run.trial080.stdout:(49/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 24 MB/s | 6.1 MB 00:00 2026-02-22T23:14:57.432 INFO:teuthology.orchestra.run.trial040.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-22T23:14:57.432 INFO:teuthology.orchestra.run.trial080.stdout:(50/99): python3-pyasn1-modules-0.4.8-6.el9.noa 3.2 MB/s | 279 kB 00:00 2026-02-22T23:14:57.437 INFO:teuthology.orchestra.run.trial040.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:57.441 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:14:57.441 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:57.441 INFO:teuthology.orchestra.run.trial189.stdout:Upgraded: 2026-02-22T23:14:57.442 INFO:teuthology.orchestra.run.trial189.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.442 INFO:teuthology.orchestra.run.trial189.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.442 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:14:57.442 INFO:teuthology.orchestra.run.trial189.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-22T23:14:57.442 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.442 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.442 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.443 INFO:teuthology.orchestra.run.trial189.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.443 INFO:teuthology.orchestra.run.trial189.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-22T23:14:57.443 INFO:teuthology.orchestra.run.trial189.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-22T23:14:57.443 INFO:teuthology.orchestra.run.trial189.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-22T23:14:57.443 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.443 INFO:teuthology.orchestra.run.trial189.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-22T23:14:57.444 INFO:teuthology.orchestra.run.trial189.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-22T23:14:57.445 INFO:teuthology.orchestra.run.trial189.stdout: re2-1:20211101-20.el9.x86_64 2026-02-22T23:14:57.446 INFO:teuthology.orchestra.run.trial189.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-22T23:14:57.446 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:57.446 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:14:57.451 INFO:teuthology.orchestra.run.trial080.stdout:(51/99): python3-requests-oauthlib-1.3.0-12.el9 669 kB/s | 54 kB 00:00 2026-02-22T23:14:57.468 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:14:57.485 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-test 2026-02-22T23:14:57.511 INFO:teuthology.orchestra.run.trial080.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 528 kB/s | 42 kB 00:00 2026-02-22T23:14:57.511 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:14:57.512 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:57.524 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:14:57.528 INFO:teuthology.orchestra.run.trial040.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-22T23:14:57.535 INFO:teuthology.orchestra.run.trial080.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.2 MB/s | 189 kB 00:00 2026-02-22T23:14:57.537 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-22T23:14:57.559 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:14:57.560 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:57.572 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:14:57.591 INFO:teuthology.orchestra.run.trial080.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.1 MB/s | 90 kB 00:00 2026-02-22T23:14:57.609 INFO:teuthology.orchestra.run.trial040.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-22T23:14:57.613 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-22T23:14:57.619 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-22T23:14:57.621 INFO:teuthology.orchestra.run.trial040.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-22T23:14:57.624 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-22T23:14:57.630 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-22T23:14:57.646 INFO:teuthology.orchestra.run.trial080.stdout:(55/99): lua-devel-5.4.4-4.el9.x86_64.rpm 201 kB/s | 22 kB 00:00 2026-02-22T23:14:57.723 INFO:teuthology.orchestra.run.trial080.stdout:(56/99): protobuf-compiler-3.14.0-17.el9.x86_64 6.4 MB/s | 862 kB 00:00 2026-02-22T23:14:57.769 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:14:57.770 INFO:teuthology.orchestra.run.trial040.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:14:57.795 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:13 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:14:57.795 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:14:57.799 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-22T23:14:57.806 INFO:teuthology.orchestra.run.trial040.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-22T23:14:57.859 INFO:teuthology.orchestra.run.trial080.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 143 kB/s | 19 kB 00:00 2026-02-22T23:14:57.877 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:14:57.878 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:14:57.878 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repository Size 2026-02-22T23:14:57.878 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:14:57.878 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:14:57.878 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-22T23:14:57.878 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout:Install 5 Packages 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 50 M 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 212 M 2026-02-22T23:14:57.879 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:14:57.912 INFO:teuthology.orchestra.run.trial080.stdout:(58/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.0 MB/s | 551 kB 00:00 2026-02-22T23:14:57.913 INFO:teuthology.orchestra.run.trial080.stdout:(59/99): luarocks-3.9.2-5.el9.noarch.rpm 2.7 MB/s | 151 kB 00:00 2026-02-22T23:14:57.971 INFO:teuthology.orchestra.run.trial080.stdout:(60/99): python3-asyncssh-2.13.2-5.el9.noarch.r 9.1 MB/s | 548 kB 00:00 2026-02-22T23:14:57.973 INFO:teuthology.orchestra.run.trial080.stdout:(61/99): python3-autocommand-2.2.2-8.el9.noarch 492 kB/s | 29 kB 00:00 2026-02-22T23:14:58.016 INFO:teuthology.orchestra.run.trial080.stdout:(62/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 33 MB/s | 19 MB 00:00 2026-02-22T23:14:58.017 INFO:teuthology.orchestra.run.trial080.stdout:(63/99): python3-backports-tarfile-1.2.0-1.el9. 1.3 MB/s | 60 kB 00:00 2026-02-22T23:14:58.017 INFO:teuthology.orchestra.run.trial080.stdout:(64/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 982 kB/s | 43 kB 00:00 2026-02-22T23:14:58.030 INFO:teuthology.orchestra.run.trial040.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-22T23:14:58.031 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:14:58.048 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:14:58.049 INFO:teuthology.orchestra.run.trial080.stdout:(65/99): python3-cachetools-4.2.4-1.el9.noarch. 997 kB/s | 32 kB 00:00 2026-02-22T23:14:58.050 INFO:teuthology.orchestra.run.trial080.stdout:(66/99): python3-certifi-2023.05.07-4.el9.noarc 433 kB/s | 14 kB 00:00 2026-02-22T23:14:58.053 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-22T23:14:58.058 INFO:teuthology.orchestra.run.trial040.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-22T23:14:58.058 INFO:teuthology.orchestra.run.trial080.stdout:(67/99): python3-cheroot-10.0.1-4.el9.noarch.rp 4.2 MB/s | 173 kB 00:00 2026-02-22T23:14:58.061 INFO:teuthology.orchestra.run.trial040.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-22T23:14:58.062 INFO:teuthology.orchestra.run.trial040.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:14:58.066 INFO:teuthology.orchestra.run.trial189.stdout:(1/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 1.8 MB/s | 163 kB 00:00 2026-02-22T23:14:58.079 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:14:58.099 INFO:teuthology.orchestra.run.trial080.stdout:(68/99): python3-cherrypy-18.6.1-2.el9.noarch.r 7.0 MB/s | 358 kB 00:00 2026-02-22T23:14:58.112 INFO:teuthology.orchestra.run.trial080.stdout:(69/99): python3-google-auth-2.45.0-1.el9.noarc 4.0 MB/s | 254 kB 00:00 2026-02-22T23:14:58.195 INFO:teuthology.orchestra.run.trial040.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-22T23:14:58.197 INFO:teuthology.orchestra.run.trial040.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:14:58.220 INFO:teuthology.orchestra.run.trial080.stdout:(70/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 13 MB/s | 2.0 MB 00:00 2026-02-22T23:14:58.226 INFO:teuthology.orchestra.run.trial080.stdout:(71/99): python3-grpcio-tools-1.46.7-10.el9.x86 1.1 MB/s | 144 kB 00:00 2026-02-22T23:14:58.233 INFO:teuthology.orchestra.run.trial080.stdout:(72/99): python3-influxdb-5.3.1-1.el9.noarch.rp 1.1 MB/s | 139 kB 00:00 2026-02-22T23:14:58.254 INFO:teuthology.orchestra.run.trial080.stdout:(73/99): python3-isodate-0.6.1-3.el9.noarch.rpm 1.7 MB/s | 56 kB 00:00 2026-02-22T23:14:58.257 INFO:teuthology.orchestra.run.trial080.stdout:(74/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 343 kB/s | 11 kB 00:00 2026-02-22T23:14:58.264 INFO:teuthology.orchestra.run.trial080.stdout:(75/99): python3-jaraco-classes-3.2.1-5.el9.noa 564 kB/s | 18 kB 00:00 2026-02-22T23:14:58.286 INFO:teuthology.orchestra.run.trial080.stdout:(76/99): python3-jaraco-collections-3.0.0-8.el9 729 kB/s | 23 kB 00:00 2026-02-22T23:14:58.289 INFO:teuthology.orchestra.run.trial080.stdout:(77/99): python3-jaraco-context-6.0.1-3.el9.noa 621 kB/s | 20 kB 00:00 2026-02-22T23:14:58.296 INFO:teuthology.orchestra.run.trial080.stdout:(78/99): python3-jaraco-functools-3.5.0-2.el9.n 621 kB/s | 19 kB 00:00 2026-02-22T23:14:58.318 INFO:teuthology.orchestra.run.trial080.stdout:(79/99): python3-jaraco-text-4.0.0-2.el9.noarch 825 kB/s | 26 kB 00:00 2026-02-22T23:14:58.373 INFO:teuthology.orchestra.run.trial080.stdout:(80/99): python3-kubernetes-26.1.0-3.el9.noarch 12 MB/s | 1.0 MB 00:00 2026-02-22T23:14:58.374 INFO:teuthology.orchestra.run.trial080.stdout:(81/99): python3-logutils-0.3.5-21.el9.noarch.r 592 kB/s | 46 kB 00:00 2026-02-22T23:14:58.379 INFO:teuthology.orchestra.run.trial080.stdout:(82/99): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2026-02-22T23:14:58.408 INFO:teuthology.orchestra.run.trial080.stdout:(83/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.4 MB/s | 86 kB 00:00 2026-02-22T23:14:58.411 INFO:teuthology.orchestra.run.trial080.stdout:(84/99): python3-natsort-7.1.1-5.el9.noarch.rpm 1.5 MB/s | 58 kB 00:00 2026-02-22T23:14:58.425 INFO:teuthology.orchestra.run.trial080.stdout:(85/99): python3-pecan-1.4.2-3.el9.noarch.rpm 5.7 MB/s | 272 kB 00:00 2026-02-22T23:14:58.439 INFO:teuthology.orchestra.run.trial080.stdout:(86/99): python3-portend-3.1.0-2.el9.noarch.rpm 517 kB/s | 16 kB 00:00 2026-02-22T23:14:58.447 INFO:teuthology.orchestra.run.trial080.stdout:(87/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.5 MB/s | 90 kB 00:00 2026-02-22T23:14:58.458 INFO:teuthology.orchestra.run.trial080.stdout:(88/99): python3-repoze-lru-0.7-16.el9.noarch.r 955 kB/s | 31 kB 00:00 2026-02-22T23:14:58.465 INFO:teuthology.orchestra.run.trial189.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 478 kB/s | 233 kB 00:00 2026-02-22T23:14:58.480 INFO:teuthology.orchestra.run.trial080.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 4.5 MB/s | 188 kB 00:00 2026-02-22T23:14:58.483 INFO:teuthology.orchestra.run.trial080.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 1.6 MB/s | 59 kB 00:00 2026-02-22T23:14:58.494 INFO:teuthology.orchestra.run.trial080.stdout:(91/99): python3-saml-1.16.0-1.el9.noarch.rpm 3.4 MB/s | 125 kB 00:00 2026-02-22T23:14:58.496 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:14:58.496 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T23:14:58.496 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:14:58.514 INFO:teuthology.orchestra.run.trial080.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 1.0 MB/s | 36 kB 00:00 2026-02-22T23:14:58.518 INFO:teuthology.orchestra.run.trial080.stdout:(93/99): python3-typing-extensions-4.15.0-1.el9 2.4 MB/s | 86 kB 00:00 2026-02-22T23:14:58.534 INFO:teuthology.orchestra.run.trial040.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-22T23:14:58.536 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-22T23:14:58.541 INFO:teuthology.orchestra.run.trial080.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 4.8 MB/s | 230 kB 00:00 2026-02-22T23:14:58.551 INFO:teuthology.orchestra.run.trial080.stdout:(95/99): python3-websocket-client-1.2.3-2.el9.n 2.4 MB/s | 90 kB 00:00 2026-02-22T23:14:58.558 INFO:teuthology.orchestra.run.trial189.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 691 kB/s | 64 kB 00:00 2026-02-22T23:14:58.572 INFO:teuthology.orchestra.run.trial080.stdout:(96/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 7.8 MB/s | 427 kB 00:00 2026-02-22T23:14:58.575 INFO:teuthology.orchestra.run.trial080.stdout:(97/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.4 MB/s | 48 kB 00:00 2026-02-22T23:14:58.582 INFO:teuthology.orchestra.run.trial080.stdout:(98/99): python3-xmltodict-0.12.0-15.el9.noarch 704 kB/s | 22 kB 00:00 2026-02-22T23:14:58.585 INFO:teuthology.orchestra.run.trial189.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 585 kB/s | 303 kB 00:00 2026-02-22T23:14:58.604 INFO:teuthology.orchestra.run.trial080.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 628 kB/s | 20 kB 00:00 2026-02-22T23:14:58.607 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:14:58.607 INFO:teuthology.orchestra.run.trial080.stdout:Total 15 MB/s | 95 MB 00:06 2026-02-22T23:14:58.930 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - CRB 1.0 MB/s | 7.8 MB 00:07 2026-02-22T23:14:58.991 INFO:teuthology.orchestra.run.trial189.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 49 MB/s | 50 MB 00:01 2026-02-22T23:14:58.992 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:14:58.993 INFO:teuthology.orchestra.run.trial189.stdout:Total 45 MB/s | 50 MB 00:01 2026-02-22T23:14:59.018 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:14:59.028 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:14:59.028 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:14:59.034 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:14:59.072 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:14:59.072 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:14:59.213 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:14:59.213 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:14:59.424 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:14:59.430 INFO:teuthology.orchestra.run.trial189.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-22T23:14:59.434 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-22T23:14:59.442 INFO:teuthology.orchestra.run.trial189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-22T23:14:59.443 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:14:59.487 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:14:59.487 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:14:59.513 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:59.515 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:59.801 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:14:59.803 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:14:59.841 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:14:59.841 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-22T23:14:59.841 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-22T23:14:59.841 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:14:59.844 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:14:59.948 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:14:59.950 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-22T23:14:59.981 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-22T23:14:59.986 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-22T23:15:00.007 INFO:teuthology.orchestra.run.trial080.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-22T23:15:00.013 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-22T23:15:00.046 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-22T23:15:00.055 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-22T23:15:00.060 INFO:teuthology.orchestra.run.trial080.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-22T23:15:00.063 INFO:teuthology.orchestra.run.trial080.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-22T23:15:00.067 INFO:teuthology.orchestra.run.trial080.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-22T23:15:00.090 INFO:teuthology.orchestra.run.trial080.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-22T23:15:00.103 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-22T23:15:00.106 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-22T23:15:00.112 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-22T23:15:00.114 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-22T23:15:00.125 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-22T23:15:00.131 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-22T23:15:00.155 INFO:teuthology.orchestra.run.trial080.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-22T23:15:00.159 INFO:teuthology.orchestra.run.trial080.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-22T23:15:00.166 INFO:teuthology.orchestra.run.trial080.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-22T23:15:00.190 INFO:teuthology.orchestra.run.trial080.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-22T23:15:00.241 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-22T23:15:00.253 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-22T23:15:00.256 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-22T23:15:00.260 INFO:teuthology.orchestra.run.trial080.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-22T23:15:00.265 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-22T23:15:00.270 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-22T23:15:00.273 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-22T23:15:00.287 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-22T23:15:00.308 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-22T23:15:00.313 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-22T23:15:00.318 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-22T23:15:00.329 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-22T23:15:00.338 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-22T23:15:00.344 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-22T23:15:00.364 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-22T23:15:00.373 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-22T23:15:00.427 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-22T23:15:00.439 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-22T23:15:00.454 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-22T23:15:00.459 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-22T23:15:00.467 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-22T23:15:00.507 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-22T23:15:00.554 INFO:teuthology.orchestra.run.trial006.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 20 kB 00:00 2026-02-22T23:15:00.829 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-22T23:15:00.839 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-22T23:15:00.843 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-22T23:15:00.848 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-22T23:15:00.850 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-22T23:15:00.878 INFO:teuthology.orchestra.run.trial080.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-22T23:15:00.921 INFO:teuthology.orchestra.run.trial080.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-22T23:15:00.928 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-22T23:15:00.931 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-22T23:15:00.937 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-22T23:15:00.940 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-22T23:15:00.947 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-22T23:15:00.951 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-22T23:15:00.980 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-22T23:15:00.992 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-22T23:15:01.026 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-22T23:15:01.083 INFO:teuthology.orchestra.run.trial006.stdout:Extra Packages for Enterprise Linux 43 MB/s | 20 MB 00:00 2026-02-22T23:15:01.274 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-22T23:15:01.298 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-22T23:15:01.302 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-22T23:15:01.361 INFO:teuthology.orchestra.run.trial080.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-22T23:15:01.363 INFO:teuthology.orchestra.run.trial080.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-22T23:15:01.384 INFO:teuthology.orchestra.run.trial080.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-22T23:15:01.690 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:15:01.708 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:15:01.709 INFO:teuthology.orchestra.run.trial080.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-22T23:15:01.783 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-22T23:15:01.940 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:15:01.941 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-22T23:15:01.941 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-22T23:15:01.941 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-22T23:15:01.941 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-22T23:15:01.995 INFO:teuthology.orchestra.run.trial189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-22T23:15:01.996 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:01.996 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:02.030 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph 2026-02-22T23:15:02.334 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:18 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:02.422 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:02.425 INFO:teuthology.orchestra.run.trial189.stdout:====================================================================================== 2026-02-22T23:15:02.425 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repository Size 2026-02-22T23:15:02.425 INFO:teuthology.orchestra.run.trial189.stdout:====================================================================================== 2026-02-22T23:15:02.425 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-22T23:15:02.426 INFO:teuthology.orchestra.run.trial189.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-22T23:15:02.427 INFO:teuthology.orchestra.run.trial189.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-22T23:15:02.428 INFO:teuthology.orchestra.run.trial189.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-22T23:15:02.429 INFO:teuthology.orchestra.run.trial189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-22T23:15:02.430 INFO:teuthology.orchestra.run.trial189.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-22T23:15:02.431 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-22T23:15:02.432 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-22T23:15:02.433 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout:Installing weak dependencies: 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-22T23:15:02.434 INFO:teuthology.orchestra.run.trial189.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-22T23:15:02.435 INFO:teuthology.orchestra.run.trial189.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-22T23:15:02.435 INFO:teuthology.orchestra.run.trial189.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-22T23:15:02.435 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:02.435 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:02.435 INFO:teuthology.orchestra.run.trial189.stdout:====================================================================================== 2026-02-22T23:15:02.435 INFO:teuthology.orchestra.run.trial189.stdout:Install 99 Packages 2026-02-22T23:15:02.435 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:02.436 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 95 M 2026-02-22T23:15:02.436 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 520 M 2026-02-22T23:15:02.437 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:02.453 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-22T23:15:02.470 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-22T23:15:02.480 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-22T23:15:02.487 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-22T23:15:02.501 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-22T23:15:02.516 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-22T23:15:02.592 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-22T23:15:02.601 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-22T23:15:02.624 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-22T23:15:02.654 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-22T23:15:02.704 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-22T23:15:02.711 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-22T23:15:02.714 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-22T23:15:02.716 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-22T23:15:02.733 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:02.734 INFO:teuthology.orchestra.run.trial080.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-22T23:15:02.734 INFO:teuthology.orchestra.run.trial080.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-22T23:15:02.734 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:02.741 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:02.766 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:02.767 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-22T23:15:02.767 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:02.800 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-22T23:15:02.863 INFO:teuthology.orchestra.run.trial080.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-22T23:15:02.866 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:02.880 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:02.880 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:02.880 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-22T23:15:02.880 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:03.202 INFO:teuthology.orchestra.run.trial189.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 32 kB/s | 6.5 kB 00:00 2026-02-22T23:15:03.578 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:03.599 INFO:teuthology.orchestra.run.trial189.stdout:(2/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 4.0 MB/s | 2.4 MB 00:00 2026-02-22T23:15:03.601 INFO:teuthology.orchestra.run.trial189.stdout:(3/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 1.8 MB/s | 1.1 MB 00:00 2026-02-22T23:15:03.604 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:03.604 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:03.604 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-22T23:15:03.604 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:03.604 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:03.604 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:03.663 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:03.665 INFO:teuthology.orchestra.run.trial080.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:03.670 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-22T23:15:03.675 INFO:teuthology.orchestra.run.trial189.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 421 kB/s | 31 kB 00:00 2026-02-22T23:15:03.690 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-22T23:15:03.692 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:04.185 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:04.188 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:04.232 INFO:teuthology.orchestra.run.trial189.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 270 kB/s | 150 kB 00:00 2026-02-22T23:15:04.271 INFO:teuthology.orchestra.run.trial189.stdout:(6/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_6 25 MB/s | 17 MB 00:00 2026-02-22T23:15:04.311 INFO:teuthology.orchestra.run.trial006.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-22T23:15:04.388 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:15:04.388 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /sys 2026-02-22T23:15:04.388 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /proc 2026-02-22T23:15:04.389 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /mnt 2026-02-22T23:15:04.389 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /var/tmp 2026-02-22T23:15:04.389 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /home 2026-02-22T23:15:04.389 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /root 2026-02-22T23:15:04.389 INFO:teuthology.orchestra.run.trial040.stdout:skipping the directory /tmp 2026-02-22T23:15:04.389 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:04.579 INFO:teuthology.orchestra.run.trial189.stdout:(7/99): ceph-mgr-diskprediction-local-19.2.3-68 24 MB/s | 7.4 MB 00:00 2026-02-22T23:15:04.609 INFO:teuthology.orchestra.run.trial189.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 726 kB/s | 22 kB 00:00 2026-02-22T23:15:04.641 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:04.642 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:04.645 INFO:teuthology.orchestra.run.trial189.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 7.0 MB/s | 253 kB 00:00 2026-02-22T23:15:04.654 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:04.655 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:04.676 INFO:teuthology.orchestra.run.trial189.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 1.5 MB/s | 49 kB 00:00 2026-02-22T23:15:04.707 INFO:teuthology.orchestra.run.trial189.stdout:(11/99): ceph-prometheus-alerts-19.2.3-689.gf90 551 kB/s | 17 kB 00:00 2026-02-22T23:15:04.712 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:04.743 INFO:teuthology.orchestra.run.trial189.stdout:(12/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 8.2 MB/s | 299 kB 00:00 2026-02-22T23:15:04.761 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-22T23:15:04.762 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:04.784 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:04.784 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:04.784 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-22T23:15:04.784 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:04.785 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:04.785 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:04.788 INFO:teuthology.orchestra.run.trial189.stdout:(13/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 17 MB/s | 769 kB 00:00 2026-02-22T23:15:04.795 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:04.837 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:15:04.861 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:15:04.861 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:04.861 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-22T23:15:04.861 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:15:04.861 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:15:04.861 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:04.862 INFO:teuthology.orchestra.run.trial040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:15:04.875 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:15:04.875 INFO:teuthology.orchestra.run.trial040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:15:04.930 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:15:04.930 INFO:teuthology.orchestra.run.trial040.stdout:Creating group 'qat' with GID 992. 2026-02-22T23:15:04.930 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:04.973 INFO:teuthology.orchestra.run.trial189.stdout:(14/99): ceph-mgr-dashboard-19.2.3-689.gf90f4ea 5.1 MB/s | 3.8 MB 00:00 2026-02-22T23:15:05.010 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:05.011 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:05.032 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:05.032 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:05.032 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-22T23:15:05.032 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:05.032 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:05.032 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:05.133 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:05.155 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:05.155 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:05.155 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-22T23:15:05.155 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:05.155 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:05.155 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:05.173 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:05.217 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-22T23:15:05.217 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-22T23:15:05.217 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-22T23:15:05.217 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-22T23:15:05.217 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-22T23:15:05.218 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-22T23:15:05.219 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-22T23:15:05.219 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:15:05.219 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-22T23:15:05.219 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-22T23:15:05.250 INFO:teuthology.orchestra.run.trial189.stdout:(15/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 145 kB/s | 40 kB 00:00 2026-02-22T23:15:05.280 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:15:05.280 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:05.280 INFO:teuthology.orchestra.run.trial040.stdout:Upgraded: 2026-02-22T23:15:05.280 INFO:teuthology.orchestra.run.trial040.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.281 INFO:teuthology.orchestra.run.trial040.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-22T23:15:05.282 INFO:teuthology.orchestra.run.trial040.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: re2-1:20211101-20.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-22T23:15:05.283 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:05.284 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:05.324 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-test 2026-02-22T23:15:05.338 INFO:teuthology.orchestra.run.trial189.stdout:(16/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 639 kB/s | 351 kB 00:00 2026-02-22T23:15:05.442 INFO:teuthology.orchestra.run.trial189.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 377 kB/s | 72 kB 00:00 2026-02-22T23:15:05.523 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:05.524 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repository Size 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout:Upgrading: 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-22T23:15:05.525 INFO:teuthology.orchestra.run.trial006.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-22T23:15:05.526 INFO:teuthology.orchestra.run.trial006.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-22T23:15:05.527 INFO:teuthology.orchestra.run.trial006.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout:Installing weak dependencies: 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout:Install 33 Packages 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout:Upgrade 2 Packages 2026-02-22T23:15:05.528 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:05.530 INFO:teuthology.orchestra.run.trial189.stdout:(18/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_ 2.0 MB/s | 4.7 MB 00:02 2026-02-22T23:15:05.530 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 63 M 2026-02-22T23:15:05.530 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:05.557 INFO:teuthology.orchestra.run.trial189.stdout:(19/99): libgfortran-11.5.0-14.el9.x86_64.rpm 3.5 MB/s | 794 kB 00:00 2026-02-22T23:15:05.636 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:12 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:05.638 INFO:teuthology.orchestra.run.trial189.stdout:(20/99): libquadmath-11.5.0-14.el9.x86_64.rpm 942 kB/s | 184 kB 00:00 2026-02-22T23:15:05.721 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repository Size 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-22T23:15:05.722 INFO:teuthology.orchestra.run.trial040.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout:Install 5 Packages 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 50 M 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 212 M 2026-02-22T23:15:05.723 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:05.729 INFO:teuthology.orchestra.run.trial189.stdout:(21/99): python3-cryptography-36.0.1-5.el9.x86_ 7.2 MB/s | 1.2 MB 00:00 2026-02-22T23:15:05.742 INFO:teuthology.orchestra.run.trial189.stdout:(22/99): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2026-02-22T23:15:05.811 INFO:teuthology.orchestra.run.trial189.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 1.6 MB/s | 135 kB 00:00 2026-02-22T23:15:05.843 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:05.843 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-22T23:15:05.843 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-22T23:15:05.843 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-22T23:15:05.843 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-22T23:15:05.843 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-22T23:15:05.844 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-22T23:15:05.845 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-22T23:15:05.846 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-22T23:15:05.847 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-22T23:15:05.848 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-22T23:15:05.849 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-22T23:15:05.850 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-22T23:15:05.851 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-22T23:15:05.851 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-22T23:15:05.857 INFO:teuthology.orchestra.run.trial189.stdout:(24/99): python3-pyparsing-2.4.7-9.el9.noarch.r 1.3 MB/s | 150 kB 00:00 2026-02-22T23:15:05.892 INFO:teuthology.orchestra.run.trial189.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 1.5 MB/s | 126 kB 00:00 2026-02-22T23:15:05.903 INFO:teuthology.orchestra.run.trial040.stdout:(1/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 1.8 MB/s | 163 kB 00:00 2026-02-22T23:15:05.906 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.907 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-22T23:15:05.908 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: lua-5.4.4-4.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-22T23:15:05.909 INFO:teuthology.orchestra.run.trial080.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-22T23:15:05.910 INFO:teuthology.orchestra.run.trial080.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-22T23:15:05.911 INFO:teuthology.orchestra.run.trial080.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-ply-3.11-14.el9.noarch 2026-02-22T23:15:05.912 INFO:teuthology.orchestra.run.trial080.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-22T23:15:05.913 INFO:teuthology.orchestra.run.trial080.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: unzip-6.0-59.el9.x86_64 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-22T23:15:05.914 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-22T23:15:05.915 INFO:teuthology.orchestra.run.trial080.stdout: zip-3.0-35.el9.x86_64 2026-02-22T23:15:05.915 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:05.915 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:05.962 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-base 2026-02-22T23:15:05.974 INFO:teuthology.orchestra.run.trial189.stdout:(26/99): unzip-6.0-59.el9.x86_64.rpm 2.2 MB/s | 182 kB 00:00 2026-02-22T23:15:05.988 INFO:teuthology.orchestra.run.trial189.stdout:(27/99): python3-urllib3-1.26.5-7.el9.noarch.rp 1.6 MB/s | 218 kB 00:00 2026-02-22T23:15:06.025 INFO:teuthology.orchestra.run.trial189.stdout:(28/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 507 kB/s | 253 kB 00:00 2026-02-22T23:15:06.059 INFO:teuthology.orchestra.run.trial189.stdout:(29/99): zip-3.0-35.el9.x86_64.rpm 3.1 MB/s | 266 kB 00:00 2026-02-22T23:15:06.274 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:34 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:06.339 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:06.358 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:06.358 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:06.358 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:06.380 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install cephadm 2026-02-22T23:15:06.407 INFO:teuthology.orchestra.run.trial189.stdout:(30/99): flexiblas-3.0.4-9.el9.x86_64.rpm 71 kB/s | 30 kB 00:00 2026-02-22T23:15:06.460 INFO:teuthology.orchestra.run.trial189.stdout:(31/99): flexiblas-openblas-openmp-3.0.4-9.el9. 37 kB/s | 15 kB 00:00 2026-02-22T23:15:06.686 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:34 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:06.751 INFO:teuthology.orchestra.run.trial080.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:06.769 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:06.770 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:06.770 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:06.777 INFO:teuthology.orchestra.run.trial189.stdout:(32/99): lua-5.4.4-4.el9.x86_64.rpm 595 kB/s | 188 kB 00:00 2026-02-22T23:15:06.791 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-immutable-object-cache 2026-02-22T23:15:06.805 INFO:teuthology.orchestra.run.trial189.stdout:(33/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 618 kB/s | 246 kB 00:00 2026-02-22T23:15:06.860 INFO:teuthology.orchestra.run.trial189.stdout:(34/99): openblas-0.3.29-1.el9.x86_64.rpm 508 kB/s | 42 kB 00:00 2026-02-22T23:15:06.869 INFO:teuthology.orchestra.run.trial040.stdout:(2/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 47 MB/s | 50 MB 00:01 2026-02-22T23:15:06.870 INFO:teuthology.orchestra.run.trial040.stdout:(3/5): socat-1.7.4.1-8.el9.x86_64.rpm 314 kB/s | 303 kB 00:00 2026-02-22T23:15:07.029 INFO:teuthology.orchestra.run.trial040.stdout:(4/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 399 kB/s | 64 kB 00:00 2026-02-22T23:15:07.045 INFO:teuthology.orchestra.run.trial006.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 5.1 MB/s | 5.5 MB 00:01 2026-02-22T23:15:07.056 INFO:teuthology.orchestra.run.trial006.stdout:(2/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 2.1 MB/s | 25 kB 00:00 2026-02-22T23:15:07.100 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:35 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:07.106 INFO:teuthology.orchestra.run.trial006.stdout:(3/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 20 MB/s | 999 kB 00:00 2026-02-22T23:15:07.140 INFO:teuthology.orchestra.run.trial006.stdout:(4/35): libradosstriper1-19.2.3-689.gf90f4eae.e 14 MB/s | 503 kB 00:00 2026-02-22T23:15:07.165 INFO:teuthology.orchestra.run.trial040.stdout:(5/5): libxslt-1.1.34-12.el9.x86_64.rpm 172 kB/s | 233 kB 00:01 2026-02-22T23:15:07.169 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:07.169 INFO:teuthology.orchestra.run.trial040.stdout:Total 35 MB/s | 50 MB 00:01 2026-02-22T23:15:07.185 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:07.185 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:07.186 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 145 k 2026-02-22T23:15:07.187 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 431 k 2026-02-22T23:15:07.187 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:07.195 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:07.205 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:07.205 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:07.281 INFO:teuthology.orchestra.run.trial006.stdout:(5/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 8.2 MB/s | 11 MB 00:01 2026-02-22T23:15:07.302 INFO:teuthology.orchestra.run.trial006.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 2.1 MB/s | 45 kB 00:00 2026-02-22T23:15:07.324 INFO:teuthology.orchestra.run.trial006.stdout:(7/35): python3-ceph-common-19.2.3-689.gf90f4ea 6.5 MB/s | 142 kB 00:00 2026-02-22T23:15:07.348 INFO:teuthology.orchestra.run.trial006.stdout:(8/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 6.7 MB/s | 165 kB 00:00 2026-02-22T23:15:07.349 INFO:teuthology.orchestra.run.trial080.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 893 kB/s | 145 kB 00:00 2026-02-22T23:15:07.349 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:07.349 INFO:teuthology.orchestra.run.trial080.stdout:Total 887 kB/s | 145 kB 00:00 2026-02-22T23:15:07.350 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:07.354 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:07.354 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:07.376 INFO:teuthology.orchestra.run.trial006.stdout:(9/35): python3-rados-19.2.3-689.gf90f4eae.el9. 11 MB/s | 322 kB 00:00 2026-02-22T23:15:07.384 INFO:teuthology.orchestra.run.trial006.stdout:(10/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_6 22 MB/s | 5.4 MB 00:00 2026-02-22T23:15:07.385 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:07.385 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:07.389 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:07.390 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:07.392 INFO:teuthology.orchestra.run.trial006.stdout:(11/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 18 MB/s | 303 kB 00:00 2026-02-22T23:15:07.401 INFO:teuthology.orchestra.run.trial006.stdout:(12/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 5.7 MB/s | 100 kB 00:00 2026-02-22T23:15:07.441 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:07.442 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:07.466 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:07.466 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:07.466 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-22T23:15:07.466 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:07.601 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:07.606 INFO:teuthology.orchestra.run.trial040.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-22T23:15:07.610 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-22T23:15:07.612 INFO:teuthology.orchestra.run.trial006.stdout:(13/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 13 MB/s | 22 MB 00:01 2026-02-22T23:15:07.618 INFO:teuthology.orchestra.run.trial040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-22T23:15:07.619 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:15:07.656 INFO:teuthology.orchestra.run.trial006.stdout:(14/35): mailcap-2.1.49-5.el9.noarch.rpm 126 kB/s | 33 kB 00:00 2026-02-22T23:15:07.678 INFO:teuthology.orchestra.run.trial006.stdout:(15/35): boost-program-options-1.75.0-13.el9.x8 4.6 MB/s | 104 kB 00:00 2026-02-22T23:15:07.688 INFO:teuthology.orchestra.run.trial006.stdout:(16/35): libnbd-1.20.3-4.el9.x86_64.rpm 17 MB/s | 164 kB 00:00 2026-02-22T23:15:07.695 INFO:teuthology.orchestra.run.trial006.stdout:(17/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 22 MB/s | 160 kB 00:00 2026-02-22T23:15:07.699 INFO:teuthology.orchestra.run.trial006.stdout:(18/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 9.9 MB/s | 45 kB 00:00 2026-02-22T23:15:07.710 INFO:teuthology.orchestra.run.trial006.stdout:(19/35): librdkafka-1.6.1-102.el9.x86_64.rpm 63 MB/s | 662 kB 00:00 2026-02-22T23:15:07.717 INFO:teuthology.orchestra.run.trial006.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 43 MB/s | 292 kB 00:00 2026-02-22T23:15:07.723 INFO:teuthology.orchestra.run.trial006.stdout:(21/35): qatlib-25.08.0-2.el9.x86_64.rpm 37 MB/s | 240 kB 00:00 2026-02-22T23:15:07.727 INFO:teuthology.orchestra.run.trial006.stdout:(22/35): qatlib-service-25.08.0-2.el9.x86_64.rp 8.2 MB/s | 37 kB 00:00 2026-02-22T23:15:07.732 INFO:teuthology.orchestra.run.trial006.stdout:(23/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 14 MB/s | 66 kB 00:00 2026-02-22T23:15:07.777 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:07.777 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:07.777 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:07.777 INFO:teuthology.orchestra.run.trial080.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:07.777 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:07.777 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:07.802 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr 2026-02-22T23:15:07.918 INFO:teuthology.orchestra.run.trial006.stdout:(24/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.6 MB/s | 308 kB 00:00 2026-02-22T23:15:08.001 INFO:teuthology.orchestra.run.trial006.stdout:(25/35): pciutils-3.7.0-7.el9.x86_64.rpm 239 kB/s | 93 kB 00:00 2026-02-22T23:15:08.035 INFO:teuthology.orchestra.run.trial189.stdout:(35/99): protobuf-3.14.0-17.el9.x86_64.rpm 875 kB/s | 1.0 MB 00:01 2026-02-22T23:15:08.078 INFO:teuthology.orchestra.run.trial006.stdout:(26/35): nvme-cli-2.16-1.el9.x86_64.rpm 1.7 MB/s | 1.2 MB 00:00 2026-02-22T23:15:08.099 INFO:teuthology.orchestra.run.trial006.stdout:(27/35): libarrow-9.0.0-15.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2026-02-22T23:15:08.102 INFO:teuthology.orchestra.run.trial006.stdout:(28/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 244 kB/s | 25 kB 00:00 2026-02-22T23:15:08.109 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:08.123 INFO:teuthology.orchestra.run.trial006.stdout:(29/35): libunwind-1.6.2-1.el9.x86_64.rpm 3.2 MB/s | 67 kB 00:00 2026-02-22T23:15:08.143 INFO:teuthology.orchestra.run.trial006.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 9.1 MB/s | 191 kB 00:00 2026-02-22T23:15:08.173 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:08.186 INFO:teuthology.orchestra.run.trial006.stdout:(31/35): thrift-0.15.0-4.el9.x86_64.rpm 37 MB/s | 1.6 MB 00:00 2026-02-22T23:15:08.192 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:08.193 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:08.193 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:08.201 INFO:teuthology.orchestra.run.trial006.stdout:(32/35): liboath-2.6.12-1.el9.x86_64.rpm 397 kB/s | 49 kB 00:00 2026-02-22T23:15:08.214 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-dashboard 2026-02-22T23:15:08.297 INFO:teuthology.orchestra.run.trial006.stdout:(33/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 33 MB/s | 3.2 MB 00:00 2026-02-22T23:15:08.314 INFO:teuthology.orchestra.run.trial006.stdout:(34/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2026-02-22T23:15:08.318 INFO:teuthology.orchestra.run.trial006.stdout:(35/35): librados2-19.2.3-689.gf90f4eae.el9.x86 26 MB/s | 3.4 MB 00:00 2026-02-22T23:15:08.321 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:08.321 INFO:teuthology.orchestra.run.trial006.stdout:Total 23 MB/s | 63 MB 00:02 2026-02-22T23:15:08.422 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:08.441 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:08.441 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:08.523 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:08.588 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:08.606 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:08.607 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:08.607 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:08.628 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-22T23:15:08.694 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:08.695 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:08.937 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:08.998 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:09.003 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:09.007 INFO:teuthology.orchestra.run.trial006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-22T23:15:09.010 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-22T23:15:09.021 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:09.022 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-22T23:15:09.022 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:09.022 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:09.042 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-rook 2026-02-22T23:15:09.173 INFO:teuthology.orchestra.run.trial006.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-22T23:15:09.177 INFO:teuthology.orchestra.run.trial006.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:15:09.208 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:15:09.256 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:15:09.257 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:15:09.269 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:15:09.273 INFO:teuthology.orchestra.run.trial006.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-22T23:15:09.283 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-22T23:15:09.308 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:15:09.308 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:15:09.320 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:15:09.351 INFO:teuthology.orchestra.run.trial006.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-22T23:15:09.352 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:37 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:09.356 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-22T23:15:09.362 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-22T23:15:09.364 INFO:teuthology.orchestra.run.trial006.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-22T23:15:09.367 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-22T23:15:09.372 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-22T23:15:09.417 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:09.435 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:09.436 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:09.436 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:09.458 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-cephadm 2026-02-22T23:15:09.528 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:15:09.529 INFO:teuthology.orchestra.run.trial006.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:15:09.555 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-22T23:15:09.558 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-22T23:15:09.565 INFO:teuthology.orchestra.run.trial006.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-22T23:15:09.689 INFO:teuthology.orchestra.run.trial189.stdout:(36/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 835 kB/s | 3.0 MB 00:03 2026-02-22T23:15:09.731 INFO:teuthology.orchestra.run.trial189.stdout:(37/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 1.8 MB/s | 5.3 MB 00:02 2026-02-22T23:15:09.769 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:37 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:09.774 INFO:teuthology.orchestra.run.trial189.stdout:(38/99): python3-devel-3.9.25-3.el9.x86_64.rpm 2.8 MB/s | 244 kB 00:00 2026-02-22T23:15:09.816 INFO:teuthology.orchestra.run.trial189.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 2.9 MB/s | 249 kB 00:00 2026-02-22T23:15:09.817 INFO:teuthology.orchestra.run.trial006.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-22T23:15:09.819 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:15:09.833 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:09.835 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-22T23:15:09.839 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-22T23:15:09.845 INFO:teuthology.orchestra.run.trial006.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-22T23:15:09.847 INFO:teuthology.orchestra.run.trial006.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-22T23:15:09.848 INFO:teuthology.orchestra.run.trial006.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:15:09.852 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:09.852 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:09.852 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:09.858 INFO:teuthology.orchestra.run.trial189.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 2.1 MB/s | 177 kB 00:00 2026-02-22T23:15:09.865 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-22T23:15:09.874 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-fuse 2026-02-22T23:15:09.915 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:15:09.933 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:15:09.942 INFO:teuthology.orchestra.run.trial189.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2026-02-22T23:15:09.981 INFO:teuthology.orchestra.run.trial006.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-22T23:15:09.982 INFO:teuthology.orchestra.run.trial189.stdout:(42/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 7.4 MB/s | 1.2 MB 00:00 2026-02-22T23:15:09.983 INFO:teuthology.orchestra.run.trial006.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:15:10.025 INFO:teuthology.orchestra.run.trial189.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 419 kB/s | 35 kB 00:00 2026-02-22T23:15:10.077 INFO:teuthology.orchestra.run.trial189.stdout:(44/99): python3-babel-2.9.1-2.el9.noarch.rpm 2.9 MB/s | 6.0 MB 00:02 2026-02-22T23:15:10.110 INFO:teuthology.orchestra.run.trial189.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 5.1 MB/s | 442 kB 00:00 2026-02-22T23:15:10.141 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:15:10.141 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-22T23:15:10.141 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-22T23:15:10.141 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-22T23:15:10.141 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-22T23:15:10.161 INFO:teuthology.orchestra.run.trial189.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 917 kB/s | 77 kB 00:00 2026-02-22T23:15:10.180 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-22T23:15:10.194 INFO:teuthology.orchestra.run.trial040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-22T23:15:10.195 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:10.195 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:10.195 INFO:teuthology.orchestra.run.trial189.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 3.1 MB/s | 267 kB 00:00 2026-02-22T23:15:10.228 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph 2026-02-22T23:15:10.245 INFO:teuthology.orchestra.run.trial189.stdout:(48/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2026-02-22T23:15:10.264 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:10.265 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:10.266 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 1.2 M 2026-02-22T23:15:10.267 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 3.6 M 2026-02-22T23:15:10.267 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:10.272 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-22T23:15:10.272 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T23:15:10.272 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:10.278 INFO:teuthology.orchestra.run.trial189.stdout:(49/99): python3-pyasn1-modules-0.4.8-6.el9.noa 3.2 MB/s | 279 kB 00:00 2026-02-22T23:15:10.307 INFO:teuthology.orchestra.run.trial006.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-22T23:15:10.309 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-22T23:15:10.329 INFO:teuthology.orchestra.run.trial189.stdout:(50/99): python3-requests-oauthlib-1.3.0-12.el9 640 kB/s | 54 kB 00:00 2026-02-22T23:15:10.387 INFO:teuthology.orchestra.run.trial080.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 9.5 MB/s | 1.2 MB 00:00 2026-02-22T23:15:10.388 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:10.388 INFO:teuthology.orchestra.run.trial080.stdout:Total 9.4 MB/s | 1.2 MB 00:00 2026-02-22T23:15:10.389 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:10.391 INFO:teuthology.orchestra.run.trial189.stdout:(51/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 15 MB/s | 6.1 MB 00:00 2026-02-22T23:15:10.392 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:10.392 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:10.413 INFO:teuthology.orchestra.run.trial189.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 501 kB/s | 42 kB 00:00 2026-02-22T23:15:10.428 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:10.429 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:10.476 INFO:teuthology.orchestra.run.trial189.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.2 MB/s | 189 kB 00:00 2026-02-22T23:15:10.486 INFO:teuthology.orchestra.run.trial189.stdout:(54/99): lua-devel-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 22 kB 00:00 2026-02-22T23:15:10.495 INFO:teuthology.orchestra.run.trial189.stdout:(55/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.1 MB/s | 90 kB 00:00 2026-02-22T23:15:10.505 INFO:teuthology.orchestra.run.trial189.stdout:(56/99): protobuf-compiler-3.14.0-17.el9.x86_64 44 MB/s | 862 kB 00:00 2026-02-22T23:15:10.530 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:10.537 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:17 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:10.548 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:10.624 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:10.627 INFO:teuthology.orchestra.run.trial040.stdout:====================================================================================== 2026-02-22T23:15:10.627 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repository Size 2026-02-22T23:15:10.627 INFO:teuthology.orchestra.run.trial040.stdout:====================================================================================== 2026-02-22T23:15:10.627 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:10.627 INFO:teuthology.orchestra.run.trial040.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-22T23:15:10.627 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-22T23:15:10.628 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-22T23:15:10.629 INFO:teuthology.orchestra.run.trial040.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-22T23:15:10.630 INFO:teuthology.orchestra.run.trial040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-22T23:15:10.631 INFO:teuthology.orchestra.run.trial040.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-22T23:15:10.632 INFO:teuthology.orchestra.run.trial040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-22T23:15:10.633 INFO:teuthology.orchestra.run.trial040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-22T23:15:10.634 INFO:teuthology.orchestra.run.trial040.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout:Installing weak dependencies: 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-22T23:15:10.635 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:10.636 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:10.636 INFO:teuthology.orchestra.run.trial040.stdout:====================================================================================== 2026-02-22T23:15:10.636 INFO:teuthology.orchestra.run.trial040.stdout:Install 99 Packages 2026-02-22T23:15:10.636 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:10.637 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 95 M 2026-02-22T23:15:10.637 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 520 M 2026-02-22T23:15:10.637 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:10.760 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:10.769 INFO:teuthology.orchestra.run.trial189.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 74 kB/s | 19 kB 00:00 2026-02-22T23:15:10.816 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:10.817 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:10.817 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:10.817 INFO:teuthology.orchestra.run.trial080.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:10.817 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:10.817 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:10.845 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-volume 2026-02-22T23:15:10.895 INFO:teuthology.orchestra.run.trial189.stdout:(58/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 31 MB/s | 19 MB 00:00 2026-02-22T23:15:11.003 INFO:teuthology.orchestra.run.trial189.stdout:(59/99): luarocks-3.9.2-5.el9.noarch.rpm 648 kB/s | 151 kB 00:00 2026-02-22T23:15:11.006 INFO:teuthology.orchestra.run.trial189.stdout:(60/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.1 MB/s | 551 kB 00:00 2026-02-22T23:15:11.081 INFO:teuthology.orchestra.run.trial189.stdout:(61/99): python3-autocommand-2.2.2-8.el9.noarch 379 kB/s | 29 kB 00:00 2026-02-22T23:15:11.084 INFO:teuthology.orchestra.run.trial189.stdout:(62/99): python3-backports-tarfile-1.2.0-1.el9. 774 kB/s | 60 kB 00:00 2026-02-22T23:15:11.157 INFO:teuthology.orchestra.run.trial189.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 442 kB/s | 32 kB 00:00 2026-02-22T23:15:11.158 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:11.167 INFO:teuthology.orchestra.run.trial189.stdout:(64/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 505 kB/s | 43 kB 00:00 2026-02-22T23:15:11.218 INFO:teuthology.orchestra.run.trial189.stdout:(65/99): python3-certifi-2023.05.07-4.el9.noarc 230 kB/s | 14 kB 00:00 2026-02-22T23:15:11.225 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:11.243 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:11.244 INFO:teuthology.orchestra.run.trial040.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 126 kB/s | 6.5 kB 00:00 2026-02-22T23:15:11.245 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:11.245 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:11.266 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librados-devel 2026-02-22T23:15:11.283 INFO:teuthology.orchestra.run.trial189.stdout:(66/99): python3-cheroot-10.0.1-4.el9.noarch.rp 1.5 MB/s | 173 kB 00:00 2026-02-22T23:15:11.291 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:15:11.294 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:15:11.329 INFO:teuthology.orchestra.run.trial040.stdout:(2/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 7.8 MB/s | 1.1 MB 00:00 2026-02-22T23:15:11.335 INFO:teuthology.orchestra.run.trial189.stdout:(67/99): python3-cherrypy-18.6.1-2.el9.noarch.r 3.0 MB/s | 358 kB 00:00 2026-02-22T23:15:11.365 INFO:teuthology.orchestra.run.trial040.stdout:(3/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 14 MB/s | 2.4 MB 00:00 2026-02-22T23:15:11.401 INFO:teuthology.orchestra.run.trial040.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 873 kB/s | 31 kB 00:00 2026-02-22T23:15:11.407 INFO:teuthology.orchestra.run.trial189.stdout:(68/99): python3-google-auth-2.45.0-1.el9.noarc 2.0 MB/s | 254 kB 00:00 2026-02-22T23:15:11.441 INFO:teuthology.orchestra.run.trial040.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 3.6 MB/s | 150 kB 00:00 2026-02-22T23:15:11.471 INFO:teuthology.orchestra.run.trial040.stdout:(6/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 21 MB/s | 4.7 MB 00:00 2026-02-22T23:15:11.499 INFO:teuthology.orchestra.run.trial189.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 1.5 MB/s | 144 kB 00:00 2026-02-22T23:15:11.538 INFO:teuthology.orchestra.run.trial189.stdout:(70/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2026-02-22T23:15:11.546 INFO:teuthology.orchestra.run.trial189.stdout:(71/99): python3-asyncssh-2.13.2-5.el9.noarch.r 842 kB/s | 548 kB 00:00 2026-02-22T23:15:11.570 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:11.578 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-22T23:15:11.582 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:15:11.592 INFO:teuthology.orchestra.run.trial189.stdout:(72/99): python3-influxdb-5.3.1-1.el9.noarch.rp 1.5 MB/s | 139 kB 00:00 2026-02-22T23:15:11.608 INFO:teuthology.orchestra.run.trial189.stdout:(73/99): python3-isodate-0.6.1-3.el9.noarch.rpm 809 kB/s | 56 kB 00:00 2026-02-22T23:15:11.624 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-22T23:15:11.625 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-22T23:15:11.625 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-22T23:15:11.625 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:11.628 INFO:teuthology.orchestra.run.trial189.stdout:(74/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 131 kB/s | 11 kB 00:00 2026-02-22T23:15:11.628 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:15:11.637 INFO:teuthology.orchestra.run.trial040.stdout:(7/99): ceph-mgr-dashboard-19.2.3-689.gf90f4eae 19 MB/s | 3.8 MB 00:00 2026-02-22T23:15:11.655 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:11.655 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:11.656 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:11.657 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 126 k 2026-02-22T23:15:11.657 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 456 k 2026-02-22T23:15:11.657 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:11.666 INFO:teuthology.orchestra.run.trial189.stdout:(75/99): python3-jaraco-classes-3.2.1-5.el9.noa 239 kB/s | 18 kB 00:00 2026-02-22T23:15:11.674 INFO:teuthology.orchestra.run.trial189.stdout:(76/99): python3-jaraco-collections-3.0.0-8.el9 348 kB/s | 23 kB 00:00 2026-02-22T23:15:11.683 INFO:teuthology.orchestra.run.trial040.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 476 kB/s | 22 kB 00:00 2026-02-22T23:15:11.709 INFO:teuthology.orchestra.run.trial189.stdout:(77/99): python3-jaraco-context-6.0.1-3.el9.noa 241 kB/s | 20 kB 00:00 2026-02-22T23:15:11.731 INFO:teuthology.orchestra.run.trial189.stdout:(78/99): python3-jaraco-functools-3.5.0-2.el9.n 303 kB/s | 19 kB 00:00 2026-02-22T23:15:11.736 INFO:teuthology.orchestra.run.trial189.stdout:(79/99): python3-jaraco-text-4.0.0-2.el9.noarch 430 kB/s | 26 kB 00:00 2026-02-22T23:15:11.736 INFO:teuthology.orchestra.run.trial040.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 4.7 MB/s | 253 kB 00:00 2026-02-22T23:15:11.788 INFO:teuthology.orchestra.run.trial040.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 956 kB/s | 49 kB 00:00 2026-02-22T23:15:11.799 INFO:teuthology.orchestra.run.trial040.stdout:(11/99): ceph-mgr-diskprediction-local-19.2.3-6 23 MB/s | 7.4 MB 00:00 2026-02-22T23:15:11.813 INFO:teuthology.orchestra.run.trial189.stdout:(80/99): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2026-02-22T23:15:11.814 INFO:teuthology.orchestra.run.trial189.stdout:(81/99): python3-logutils-0.3.5-21.el9.noarch.r 557 kB/s | 46 kB 00:00 2026-02-22T23:15:11.816 INFO:teuthology.orchestra.run.trial040.stdout:(12/99): ceph-prometheus-alerts-19.2.3-689.gf90 585 kB/s | 17 kB 00:00 2026-02-22T23:15:11.831 INFO:teuthology.orchestra.run.trial040.stdout:(13/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 9.4 MB/s | 299 kB 00:00 2026-02-22T23:15:11.863 INFO:teuthology.orchestra.run.trial040.stdout:(14/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 16 MB/s | 769 kB 00:00 2026-02-22T23:15:11.888 INFO:teuthology.orchestra.run.trial189.stdout:(82/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2026-02-22T23:15:11.905 INFO:teuthology.orchestra.run.trial189.stdout:(83/99): python3-natsort-7.1.1-5.el9.noarch.rpm 634 kB/s | 58 kB 00:00 2026-02-22T23:15:11.912 INFO:teuthology.orchestra.run.trial189.stdout:(84/99): python3-kubernetes-26.1.0-3.el9.noarch 5.0 MB/s | 1.0 MB 00:00 2026-02-22T23:15:11.918 INFO:teuthology.orchestra.run.trial080.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 482 kB/s | 126 kB 00:00 2026-02-22T23:15:11.919 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:11.919 INFO:teuthology.orchestra.run.trial080.stdout:Total 480 kB/s | 126 kB 00:00 2026-02-22T23:15:11.919 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:11.922 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:11.922 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:11.949 INFO:teuthology.orchestra.run.trial040.stdout:(15/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_ 28 MB/s | 17 MB 00:00 2026-02-22T23:15:11.954 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:11.954 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:11.974 INFO:teuthology.orchestra.run.trial189.stdout:(85/99): python3-pecan-1.4.2-3.el9.noarch.rpm 3.1 MB/s | 272 kB 00:00 2026-02-22T23:15:11.978 INFO:teuthology.orchestra.run.trial189.stdout:(86/99): python3-portend-3.1.0-2.el9.noarch.rpm 223 kB/s | 16 kB 00:00 2026-02-22T23:15:12.005 INFO:teuthology.orchestra.run.trial189.stdout:(87/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 974 kB/s | 90 kB 00:00 2026-02-22T23:15:12.009 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:12.026 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:12.037 INFO:teuthology.orchestra.run.trial189.stdout:(88/99): python3-repoze-lru-0.7-16.el9.noarch.r 482 kB/s | 31 kB 00:00 2026-02-22T23:15:12.077 INFO:teuthology.orchestra.run.trial189.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 1.9 MB/s | 188 kB 00:00 2026-02-22T23:15:12.087 INFO:teuthology.orchestra.run.trial189.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 719 kB/s | 59 kB 00:00 2026-02-22T23:15:12.105 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:12.109 INFO:teuthology.orchestra.run.trial189.stdout:(91/99): python3-saml-1.16.0-1.el9.noarch.rpm 1.7 MB/s | 125 kB 00:00 2026-02-22T23:15:12.154 INFO:teuthology.orchestra.run.trial189.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 465 kB/s | 36 kB 00:00 2026-02-22T23:15:12.162 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:12.162 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:12.162 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:12.162 INFO:teuthology.orchestra.run.trial080.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:12.162 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:12.162 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:12.181 INFO:teuthology.orchestra.run.trial189.stdout:(93/99): python3-typing-extensions-4.15.0-1.el9 921 kB/s | 86 kB 00:00 2026-02-22T23:15:12.187 INFO:teuthology.orchestra.run.trial189.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 2.9 MB/s | 230 kB 00:00 2026-02-22T23:15:12.189 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install libcephfs2 2026-02-22T23:15:12.229 INFO:teuthology.orchestra.run.trial189.stdout:(95/99): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2026-02-22T23:15:12.264 INFO:teuthology.orchestra.run.trial189.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 635 kB/s | 48 kB 00:00 2026-02-22T23:15:12.287 INFO:teuthology.orchestra.run.trial189.stdout:(97/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.9 MB/s | 427 kB 00:00 2026-02-22T23:15:12.293 INFO:teuthology.orchestra.run.trial189.stdout:(98/99): python3-xmltodict-0.12.0-15.el9.noarch 343 kB/s | 22 kB 00:00 2026-02-22T23:15:12.333 INFO:teuthology.orchestra.run.trial189.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 291 kB/s | 20 kB 00:00 2026-02-22T23:15:12.337 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:12.337 INFO:teuthology.orchestra.run.trial189.stdout:Total 9.6 MB/s | 95 MB 00:09 2026-02-22T23:15:12.474 INFO:teuthology.orchestra.run.trial040.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 66 kB/s | 40 kB 00:00 2026-02-22T23:15:12.505 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:40 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:12.572 INFO:teuthology.orchestra.run.trial080.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:12.589 INFO:teuthology.orchestra.run.trial040.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 113 kB/s | 72 kB 00:00 2026-02-22T23:15:12.591 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:12.591 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:12.591 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:12.612 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install libcephfs-devel 2026-02-22T23:15:12.769 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:12.807 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:12.807 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:12.821 INFO:teuthology.orchestra.run.trial040.stdout:(18/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 354 kB/s | 351 kB 00:00 2026-02-22T23:15:12.904 INFO:teuthology.orchestra.run.trial040.stdout:(19/99): libquadmath-11.5.0-14.el9.x86_64.rpm 585 kB/s | 184 kB 00:00 2026-02-22T23:15:12.916 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:40 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:13.002 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:13.002 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:13.002 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:15:13.002 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:13.002 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:13.003 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-22T23:15:13.003 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:13.003 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:13.003 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:13.003 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:13.004 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:13.004 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 34 k 2026-02-22T23:15:13.004 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 153 k 2026-02-22T23:15:13.004 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:13.067 INFO:teuthology.orchestra.run.trial080.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 514 kB/s | 34 kB 00:00 2026-02-22T23:15:13.067 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:13.067 INFO:teuthology.orchestra.run.trial080.stdout:Total 506 kB/s | 34 kB 00:00 2026-02-22T23:15:13.067 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:13.069 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:13.069 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:13.072 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:13.072 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:13.094 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:13.111 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:13.167 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:13.216 INFO:teuthology.orchestra.run.trial040.stdout:(20/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 641 kB/s | 253 kB 00:00 2026-02-22T23:15:13.223 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:13.223 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:13.223 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:13.223 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:13.224 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:13.224 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:13.224 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:13.224 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:13.251 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librados2 2026-02-22T23:15:13.454 INFO:teuthology.orchestra.run.trial040.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 449 kB/s | 106 kB 00:00 2026-02-22T23:15:13.500 INFO:teuthology.orchestra.run.trial040.stdout:(22/99): libgfortran-11.5.0-14.el9.x86_64.rpm 774 kB/s | 794 kB 00:01 2026-02-22T23:15:13.560 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:41 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:13.625 INFO:teuthology.orchestra.run.trial080.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:13.643 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:13.643 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:13.643 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:13.665 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librbd1 2026-02-22T23:15:13.692 INFO:teuthology.orchestra.run.trial040.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 569 kB/s | 135 kB 00:00 2026-02-22T23:15:13.698 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:13.700 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-22T23:15:13.730 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-22T23:15:13.736 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-22T23:15:13.757 INFO:teuthology.orchestra.run.trial189.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-22T23:15:13.763 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-22T23:15:13.795 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-22T23:15:13.804 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-22T23:15:13.810 INFO:teuthology.orchestra.run.trial189.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-22T23:15:13.812 INFO:teuthology.orchestra.run.trial189.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-22T23:15:13.816 INFO:teuthology.orchestra.run.trial189.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-22T23:15:13.817 INFO:teuthology.orchestra.run.trial040.stdout:(24/99): python3-pyparsing-2.4.7-9.el9.noarch.r 475 kB/s | 150 kB 00:00 2026-02-22T23:15:13.839 INFO:teuthology.orchestra.run.trial189.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-22T23:15:13.851 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-22T23:15:13.854 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-22T23:15:13.861 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-22T23:15:13.862 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-22T23:15:13.873 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-22T23:15:13.879 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-22T23:15:13.903 INFO:teuthology.orchestra.run.trial189.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-22T23:15:13.906 INFO:teuthology.orchestra.run.trial189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-22T23:15:13.913 INFO:teuthology.orchestra.run.trial189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-22T23:15:13.929 INFO:teuthology.orchestra.run.trial040.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 532 kB/s | 126 kB 00:00 2026-02-22T23:15:13.937 INFO:teuthology.orchestra.run.trial189.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-22T23:15:13.979 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:41 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:13.987 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-22T23:15:14.000 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-22T23:15:14.002 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-22T23:15:14.007 INFO:teuthology.orchestra.run.trial189.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-22T23:15:14.012 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-22T23:15:14.017 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-22T23:15:14.020 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-22T23:15:14.034 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-22T23:15:14.044 INFO:teuthology.orchestra.run.trial080.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:14.054 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-22T23:15:14.059 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-22T23:15:14.062 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:14.063 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:14.063 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:14.064 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-22T23:15:14.075 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-22T23:15:14.084 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-22T23:15:14.085 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rados 2026-02-22T23:15:14.090 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-22T23:15:14.109 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-22T23:15:14.118 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-22T23:15:14.171 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-22T23:15:14.183 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-22T23:15:14.199 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-22T23:15:14.203 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-22T23:15:14.211 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-22T23:15:14.212 INFO:teuthology.orchestra.run.trial040.stdout:(26/99): python3-urllib3-1.26.5-7.el9.noarch.rp 551 kB/s | 218 kB 00:00 2026-02-22T23:15:14.245 INFO:teuthology.orchestra.run.trial040.stdout:(27/99): unzip-6.0-59.el9.x86_64.rpm 576 kB/s | 182 kB 00:00 2026-02-22T23:15:14.251 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-22T23:15:14.335 INFO:teuthology.orchestra.run.trial040.stdout:(28/99): flexiblas-3.0.4-9.el9.x86_64.rpm 328 kB/s | 30 kB 00:00 2026-02-22T23:15:14.392 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:14.458 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:14.476 INFO:teuthology.orchestra.run.trial040.stdout:(29/99): python3-cryptography-36.0.1-5.el9.x86_ 811 kB/s | 1.2 MB 00:01 2026-02-22T23:15:14.477 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:14.477 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:14.478 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:14.500 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rgw 2026-02-22T23:15:14.540 INFO:teuthology.orchestra.run.trial040.stdout:(30/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 15 MB/s | 3.0 MB 00:00 2026-02-22T23:15:14.550 INFO:teuthology.orchestra.run.trial040.stdout:(31/99): flexiblas-openblas-openmp-3.0.4-9.el9. 202 kB/s | 15 kB 00:00 2026-02-22T23:15:14.567 INFO:teuthology.orchestra.run.trial040.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 8.7 MB/s | 246 kB 00:00 2026-02-22T23:15:14.575 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-22T23:15:14.585 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-22T23:15:14.589 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-22T23:15:14.593 INFO:teuthology.orchestra.run.trial040.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 1.6 MB/s | 42 kB 00:00 2026-02-22T23:15:14.594 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-22T23:15:14.596 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-22T23:15:14.606 INFO:teuthology.orchestra.run.trial040.stdout:(34/99): zip-3.0-35.el9.x86_64.rpm 675 kB/s | 266 kB 00:00 2026-02-22T23:15:14.624 INFO:teuthology.orchestra.run.trial189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-22T23:15:14.659 INFO:teuthology.orchestra.run.trial040.stdout:(35/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 80 MB/s | 5.3 MB 00:00 2026-02-22T23:15:14.665 INFO:teuthology.orchestra.run.trial040.stdout:(36/99): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2026-02-22T23:15:14.667 INFO:teuthology.orchestra.run.trial189.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-22T23:15:14.673 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-22T23:15:14.677 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-22T23:15:14.683 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-22T23:15:14.686 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-22T23:15:14.693 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-22T23:15:14.696 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-22T23:15:14.724 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-22T23:15:14.734 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-22T23:15:14.734 INFO:teuthology.orchestra.run.trial040.stdout:(37/99): python3-devel-3.9.25-3.el9.x86_64.rpm 3.4 MB/s | 244 kB 00:00 2026-02-22T23:15:14.746 INFO:teuthology.orchestra.run.trial040.stdout:(38/99): python3-babel-2.9.1-2.el9.noarch.rpm 69 MB/s | 6.0 MB 00:00 2026-02-22T23:15:14.768 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-22T23:15:14.768 INFO:teuthology.orchestra.run.trial040.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 7.2 MB/s | 249 kB 00:00 2026-02-22T23:15:14.774 INFO:teuthology.orchestra.run.trial040.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 6.2 MB/s | 177 kB 00:00 2026-02-22T23:15:14.802 INFO:teuthology.orchestra.run.trial040.stdout:(41/99): python3-mako-1.1.4-6.el9.noarch.rpm 6.0 MB/s | 172 kB 00:00 2026-02-22T23:15:14.812 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:14.813 INFO:teuthology.orchestra.run.trial040.stdout:(42/99): protobuf-3.14.0-17.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2026-02-22T23:15:14.830 INFO:teuthology.orchestra.run.trial040.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 1.2 MB/s | 35 kB 00:00 2026-02-22T23:15:14.840 INFO:teuthology.orchestra.run.trial040.stdout:(44/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 17 MB/s | 1.2 MB 00:00 2026-02-22T23:15:14.858 INFO:teuthology.orchestra.run.trial040.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 16 MB/s | 442 kB 00:00 2026-02-22T23:15:14.867 INFO:teuthology.orchestra.run.trial040.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 2.8 MB/s | 77 kB 00:00 2026-02-22T23:15:14.877 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:14.887 INFO:teuthology.orchestra.run.trial040.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 9.2 MB/s | 267 kB 00:00 2026-02-22T23:15:14.895 INFO:teuthology.orchestra.run.trial040.stdout:(48/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 5.5 MB/s | 159 kB 00:00 2026-02-22T23:15:14.896 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:14.896 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:14.896 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:14.918 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-cephfs 2026-02-22T23:15:14.921 INFO:teuthology.orchestra.run.trial040.stdout:(49/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 57 MB/s | 6.1 MB 00:00 2026-02-22T23:15:14.922 INFO:teuthology.orchestra.run.trial040.stdout:(50/99): python3-pyasn1-modules-0.4.8-6.el9.noa 7.8 MB/s | 279 kB 00:00 2026-02-22T23:15:14.922 INFO:teuthology.orchestra.run.trial040.stdout:(51/99): python3-requests-oauthlib-1.3.0-12.el9 1.9 MB/s | 54 kB 00:00 2026-02-22T23:15:14.949 INFO:teuthology.orchestra.run.trial040.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 42 kB 00:00 2026-02-22T23:15:14.952 INFO:teuthology.orchestra.run.trial040.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 6.4 MB/s | 189 kB 00:00 2026-02-22T23:15:14.976 INFO:teuthology.orchestra.run.trial040.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.3 MB/s | 90 kB 00:00 2026-02-22T23:15:14.992 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-22T23:15:15.015 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-22T23:15:15.019 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-22T23:15:15.043 INFO:teuthology.orchestra.run.trial040.stdout:(55/99): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2026-02-22T23:15:15.077 INFO:teuthology.orchestra.run.trial189.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-22T23:15:15.078 INFO:teuthology.orchestra.run.trial189.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-22T23:15:15.100 INFO:teuthology.orchestra.run.trial189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-22T23:15:15.103 INFO:teuthology.orchestra.run.trial040.stdout:(56/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 106 MB/s | 19 MB 00:00 2026-02-22T23:15:15.208 INFO:teuthology.orchestra.run.trial040.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 187 kB/s | 19 kB 00:00 2026-02-22T23:15:15.225 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:15.285 INFO:teuthology.orchestra.run.trial040.stdout:(58/99): luarocks-3.9.2-5.el9.noarch.rpm 1.9 MB/s | 151 kB 00:00 2026-02-22T23:15:15.292 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:15.295 INFO:teuthology.orchestra.run.trial040.stdout:(59/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.1 MB/s | 551 kB 00:00 2026-02-22T23:15:15.310 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:15.311 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:15.311 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:15.315 INFO:teuthology.orchestra.run.trial040.stdout:(60/99): python3-autocommand-2.2.2-8.el9.noarch 1.5 MB/s | 29 kB 00:00 2026-02-22T23:15:15.332 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rbd 2026-02-22T23:15:15.335 INFO:teuthology.orchestra.run.trial040.stdout:(61/99): python3-backports-tarfile-1.2.0-1.el9. 2.9 MB/s | 60 kB 00:00 2026-02-22T23:15:15.355 INFO:teuthology.orchestra.run.trial040.stdout:(62/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.1 MB/s | 43 kB 00:00 2026-02-22T23:15:15.376 INFO:teuthology.orchestra.run.trial040.stdout:(63/99): python3-cachetools-4.2.4-1.el9.noarch. 1.5 MB/s | 32 kB 00:00 2026-02-22T23:15:15.396 INFO:teuthology.orchestra.run.trial040.stdout:(64/99): python3-certifi-2023.05.07-4.el9.noarc 721 kB/s | 14 kB 00:00 2026-02-22T23:15:15.420 INFO:teuthology.orchestra.run.trial040.stdout:(65/99): python3-asyncssh-2.13.2-5.el9.noarch.r 4.0 MB/s | 548 kB 00:00 2026-02-22T23:15:15.423 INFO:teuthology.orchestra.run.trial189.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-22T23:15:15.454 INFO:teuthology.orchestra.run.trial040.stdout:(66/99): python3-cheroot-10.0.1-4.el9.noarch.rp 2.9 MB/s | 173 kB 00:00 2026-02-22T23:15:15.494 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-22T23:15:15.498 INFO:teuthology.orchestra.run.trial040.stdout:(67/99): python3-cherrypy-18.6.1-2.el9.noarch.r 4.5 MB/s | 358 kB 00:00 2026-02-22T23:15:15.531 INFO:teuthology.orchestra.run.trial040.stdout:(68/99): python3-google-auth-2.45.0-1.el9.noarc 3.2 MB/s | 254 kB 00:00 2026-02-22T23:15:15.570 INFO:teuthology.orchestra.run.trial040.stdout:(69/99): python3-grpcio-tools-1.46.7-10.el9.x86 3.6 MB/s | 144 kB 00:00 2026-02-22T23:15:15.609 INFO:teuthology.orchestra.run.trial040.stdout:(70/99): python3-influxdb-5.3.1-1.el9.noarch.rp 3.5 MB/s | 139 kB 00:00 2026-02-22T23:15:15.629 INFO:teuthology.orchestra.run.trial040.stdout:(71/99): python3-isodate-0.6.1-3.el9.noarch.rpm 2.7 MB/s | 56 kB 00:00 2026-02-22T23:15:15.637 INFO:teuthology.orchestra.run.trial040.stdout:(72/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 15 MB/s | 2.0 MB 00:00 2026-02-22T23:15:15.640 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:15.649 INFO:teuthology.orchestra.run.trial040.stdout:(73/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 538 kB/s | 11 kB 00:00 2026-02-22T23:15:15.658 INFO:teuthology.orchestra.run.trial040.stdout:(74/99): python3-jaraco-classes-3.2.1-5.el9.noa 879 kB/s | 18 kB 00:00 2026-02-22T23:15:15.669 INFO:teuthology.orchestra.run.trial040.stdout:(75/99): python3-jaraco-collections-3.0.0-8.el9 1.2 MB/s | 23 kB 00:00 2026-02-22T23:15:15.678 INFO:teuthology.orchestra.run.trial040.stdout:(76/99): python3-jaraco-context-6.0.1-3.el9.noa 983 kB/s | 20 kB 00:00 2026-02-22T23:15:15.688 INFO:teuthology.orchestra.run.trial040.stdout:(77/99): python3-jaraco-functools-3.5.0-2.el9.n 994 kB/s | 19 kB 00:00 2026-02-22T23:15:15.698 INFO:teuthology.orchestra.run.trial040.stdout:(78/99): python3-jaraco-text-4.0.0-2.el9.noarch 1.3 MB/s | 26 kB 00:00 2026-02-22T23:15:15.707 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:15.718 INFO:teuthology.orchestra.run.trial040.stdout:(79/99): python3-logutils-0.3.5-21.el9.noarch.r 2.3 MB/s | 46 kB 00:00 2026-02-22T23:15:15.725 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:15.726 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:15:15.726 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:15.739 INFO:teuthology.orchestra.run.trial040.stdout:(80/99): python3-more-itertools-8.12.0-2.el9.no 3.7 MB/s | 79 kB 00:00 2026-02-22T23:15:15.747 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-fuse 2026-02-22T23:15:15.759 INFO:teuthology.orchestra.run.trial040.stdout:(81/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 4.2 MB/s | 86 kB 00:00 2026-02-22T23:15:15.779 INFO:teuthology.orchestra.run.trial040.stdout:(82/99): python3-natsort-7.1.1-5.el9.noarch.rpm 2.8 MB/s | 58 kB 00:00 2026-02-22T23:15:15.800 INFO:teuthology.orchestra.run.trial040.stdout:(83/99): python3-pecan-1.4.2-3.el9.noarch.rpm 13 MB/s | 272 kB 00:00 2026-02-22T23:15:15.820 INFO:teuthology.orchestra.run.trial040.stdout:(84/99): python3-portend-3.1.0-2.el9.noarch.rpm 824 kB/s | 16 kB 00:00 2026-02-22T23:15:15.841 INFO:teuthology.orchestra.run.trial040.stdout:(85/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 4.3 MB/s | 90 kB 00:00 2026-02-22T23:15:15.861 INFO:teuthology.orchestra.run.trial040.stdout:(86/99): python3-repoze-lru-0.7-16.el9.noarch.r 1.5 MB/s | 31 kB 00:00 2026-02-22T23:15:15.881 INFO:teuthology.orchestra.run.trial040.stdout:(87/99): python3-routes-2.5.1-5.el9.noarch.rpm 9.0 MB/s | 188 kB 00:00 2026-02-22T23:15:15.902 INFO:teuthology.orchestra.run.trial040.stdout:(88/99): python3-rsa-4.9-2.el9.noarch.rpm 2.9 MB/s | 59 kB 00:00 2026-02-22T23:15:15.918 INFO:teuthology.orchestra.run.trial040.stdout:(89/99): python3-kubernetes-26.1.0-3.el9.noarch 4.4 MB/s | 1.0 MB 00:00 2026-02-22T23:15:15.922 INFO:teuthology.orchestra.run.trial040.stdout:(90/99): python3-saml-1.16.0-1.el9.noarch.rpm 5.9 MB/s | 125 kB 00:00 2026-02-22T23:15:15.939 INFO:teuthology.orchestra.run.trial040.stdout:(91/99): python3-tempora-5.0.0-2.el9.noarch.rpm 1.7 MB/s | 36 kB 00:00 2026-02-22T23:15:15.942 INFO:teuthology.orchestra.run.trial040.stdout:(92/99): python3-typing-extensions-4.15.0-1.el9 4.2 MB/s | 86 kB 00:00 2026-02-22T23:15:15.963 INFO:teuthology.orchestra.run.trial040.stdout:(93/99): python3-websocket-client-1.2.3-2.el9.n 4.3 MB/s | 90 kB 00:00 2026-02-22T23:15:15.985 INFO:teuthology.orchestra.run.trial040.stdout:(94/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 19 MB/s | 427 kB 00:00 2026-02-22T23:15:15.997 INFO:teuthology.orchestra.run.trial040.stdout:(95/99): python3-webob-1.8.8-2.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2026-02-22T23:15:16.005 INFO:teuthology.orchestra.run.trial040.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 2.3 MB/s | 48 kB 00:00 2026-02-22T23:15:16.017 INFO:teuthology.orchestra.run.trial040.stdout:(97/99): python3-xmltodict-0.12.0-15.el9.noarch 1.1 MB/s | 22 kB 00:00 2026-02-22T23:15:16.025 INFO:teuthology.orchestra.run.trial040.stdout:(98/99): python3-zc-lockfile-2.0-10.el9.noarch. 998 kB/s | 20 kB 00:00 2026-02-22T23:15:16.050 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:44 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:16.085 INFO:teuthology.orchestra.run.trial040.stdout:(99/99): protobuf-compiler-3.14.0-17.el9.x86_64 777 kB/s | 862 kB 00:01 2026-02-22T23:15:16.091 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:16.091 INFO:teuthology.orchestra.run.trial040.stdout:Total 17 MB/s | 95 MB 00:05 2026-02-22T23:15:16.135 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:16.135 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:16.135 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 85 k 2026-02-22T23:15:16.136 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 227 k 2026-02-22T23:15:16.137 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:16.154 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /sys 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /proc 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /mnt 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /var/tmp 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /home 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /root 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout:skipping the directory /tmp 2026-02-22T23:15:16.161 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:16.171 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-22T23:15:16.181 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-22T23:15:16.187 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-22T23:15:16.200 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-22T23:15:16.210 INFO:teuthology.orchestra.run.trial080.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-22T23:15:16.210 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:16.210 INFO:teuthology.orchestra.run.trial080.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-22T23:15:16.210 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:16.214 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:16.214 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:16.216 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-22T23:15:16.245 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:16.245 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:16.290 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-22T23:15:16.295 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:16.299 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-22T23:15:16.312 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:16.322 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-22T23:15:16.351 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-22T23:15:16.390 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:16.400 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-22T23:15:16.407 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-22T23:15:16.410 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-22T23:15:16.411 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-22T23:15:16.430 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:16.431 INFO:teuthology.orchestra.run.trial189.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-22T23:15:16.431 INFO:teuthology.orchestra.run.trial189.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-22T23:15:16.431 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:16.438 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:16.448 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:16.448 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:16.448 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:16.448 INFO:teuthology.orchestra.run.trial080.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:16.448 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:16.448 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:16.463 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:16.463 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-22T23:15:16.464 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:16.477 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-mirror 2026-02-22T23:15:16.498 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-22T23:15:16.512 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:16.551 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:16.551 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:16.561 INFO:teuthology.orchestra.run.trial189.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-22T23:15:16.563 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:16.577 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:16.577 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:16.577 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-22T23:15:16.577 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:16.611 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:15:16.636 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-22T23:15:16.636 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:16.636 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-22T23:15:16.636 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:15:16.636 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:15:16.636 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:16.638 INFO:teuthology.orchestra.run.trial006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:15:16.650 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-22T23:15:16.650 INFO:teuthology.orchestra.run.trial006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:15:16.705 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:15:16.705 INFO:teuthology.orchestra.run.trial006.stdout:Creating group 'qat' with GID 992. 2026-02-22T23:15:16.705 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:16.787 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:44 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:16.871 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 3.1 M 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 13 M 2026-02-22T23:15:16.872 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:16.969 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:16.969 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:16.996 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-22T23:15:16.996 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-22T23:15:16.996 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-22T23:15:16.996 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-22T23:15:16.997 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-22T23:15:17.023 INFO:teuthology.orchestra.run.trial080.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 21 MB/s | 3.1 MB 00:00 2026-02-22T23:15:17.023 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:17.024 INFO:teuthology.orchestra.run.trial080.stdout:Total 21 MB/s | 3.1 MB 00:00 2026-02-22T23:15:17.024 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:17.028 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:17.028 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout:Upgraded: 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.061 INFO:teuthology.orchestra.run.trial006.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: re2-1:20211101-20.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:17.062 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:17.068 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:17.068 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:17.105 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-test 2026-02-22T23:15:17.262 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:17.264 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:17.276 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:17.287 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:17.287 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:17.287 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-22T23:15:17.287 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:17.287 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:17.287 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:17.301 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:17.301 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:17.301 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-22T23:15:17.301 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:17.301 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:17.301 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:17.358 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:17.360 INFO:teuthology.orchestra.run.trial189.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:17.365 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-22T23:15:17.384 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-22T23:15:17.386 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:17.415 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:13 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:17.432 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:17.434 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-22T23:15:17.465 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-22T23:15:17.471 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-22T23:15:17.492 INFO:teuthology.orchestra.run.trial040.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-22T23:15:17.498 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repository Size 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-22T23:15:17.499 INFO:teuthology.orchestra.run.trial006.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-22T23:15:17.500 INFO:teuthology.orchestra.run.trial006.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-22T23:15:17.500 INFO:teuthology.orchestra.run.trial006.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-22T23:15:17.500 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:17.500 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:17.500 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:17.500 INFO:teuthology.orchestra.run.trial006.stdout:Install 5 Packages 2026-02-22T23:15:17.500 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:17.501 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 50 M 2026-02-22T23:15:17.502 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 212 M 2026-02-22T23:15:17.502 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:17.531 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-22T23:15:17.541 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-22T23:15:17.546 INFO:teuthology.orchestra.run.trial040.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-22T23:15:17.548 INFO:teuthology.orchestra.run.trial040.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-22T23:15:17.553 INFO:teuthology.orchestra.run.trial040.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-22T23:15:17.576 INFO:teuthology.orchestra.run.trial040.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-22T23:15:17.587 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:17.587 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:17.587 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:17.587 INFO:teuthology.orchestra.run.trial080.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:17.587 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:17.587 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:17.589 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-22T23:15:17.592 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-22T23:15:17.598 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-22T23:15:17.600 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-22T23:15:17.611 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-22T23:15:17.615 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-nbd 2026-02-22T23:15:17.617 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-22T23:15:17.642 INFO:teuthology.orchestra.run.trial040.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-22T23:15:17.645 INFO:teuthology.orchestra.run.trial040.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-22T23:15:17.652 INFO:teuthology.orchestra.run.trial040.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-22T23:15:17.676 INFO:teuthology.orchestra.run.trial040.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-22T23:15:17.711 INFO:teuthology.orchestra.run.trial006.stdout:(1/5): libxslt-1.1.34-12.el9.x86_64.rpm 2.6 MB/s | 233 kB 00:00 2026-02-22T23:15:17.712 INFO:teuthology.orchestra.run.trial006.stdout:(2/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 1.8 MB/s | 163 kB 00:00 2026-02-22T23:15:17.727 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-22T23:15:17.740 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-22T23:15:17.742 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-22T23:15:17.747 INFO:teuthology.orchestra.run.trial040.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-22T23:15:17.752 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-22T23:15:17.757 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-22T23:15:17.758 INFO:teuthology.orchestra.run.trial006.stdout:(3/5): socat-1.7.4.1-8.el9.x86_64.rpm 6.3 MB/s | 303 kB 00:00 2026-02-22T23:15:17.760 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-22T23:15:17.774 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-22T23:15:17.775 INFO:teuthology.orchestra.run.trial006.stdout:(4/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 64 kB 00:00 2026-02-22T23:15:17.795 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-22T23:15:17.800 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-22T23:15:17.805 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-22T23:15:17.816 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-22T23:15:17.825 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-22T23:15:17.831 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-22T23:15:17.851 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-22T23:15:17.860 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-22T23:15:17.876 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:17.880 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:17.914 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-22T23:15:17.926 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-22T23:15:17.927 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:45 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:17.942 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-22T23:15:17.946 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-22T23:15:17.954 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-22T23:15:17.994 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-22T23:15:18.010 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:18.011 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:18.011 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:18.011 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 170 k 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 490 k 2026-02-22T23:15:18.012 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:18.318 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-22T23:15:18.328 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-22T23:15:18.332 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-22T23:15:18.333 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:18.334 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:18.338 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-22T23:15:18.339 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-22T23:15:18.348 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:18.348 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:18.367 INFO:teuthology.orchestra.run.trial040.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-22T23:15:18.404 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:18.410 INFO:teuthology.orchestra.run.trial040.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-22T23:15:18.415 INFO:teuthology.orchestra.run.trial080.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 423 kB/s | 170 kB 00:00 2026-02-22T23:15:18.415 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:18.415 INFO:teuthology.orchestra.run.trial080.stdout:Total 421 kB/s | 170 kB 00:00 2026-02-22T23:15:18.415 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:18.417 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-22T23:15:18.419 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:18.419 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:18.420 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-22T23:15:18.426 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-22T23:15:18.430 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-22T23:15:18.437 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-22T23:15:18.440 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-22T23:15:18.451 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:18.451 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:18.453 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-22T23:15:18.454 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:18.468 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-22T23:15:18.475 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:18.475 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:18.475 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-22T23:15:18.475 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:18.475 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:18.475 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:18.478 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-22T23:15:18.486 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:18.507 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:18.512 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-22T23:15:18.523 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:18.603 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:18.639 INFO:teuthology.orchestra.run.trial006.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 49 MB/s | 50 MB 00:01 2026-02-22T23:15:18.640 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:18.640 INFO:teuthology.orchestra.run.trial006.stdout:Total 44 MB/s | 50 MB 00:01 2026-02-22T23:15:18.660 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:18.660 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:18.661 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:18.661 INFO:teuthology.orchestra.run.trial080.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:18.661 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:18.661 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:18.667 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:18.676 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:18.676 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:18.691 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:15:18.700 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:18.701 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:18.722 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:18.722 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:18.722 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-22T23:15:18.722 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:18.722 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:18.722 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:18.736 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-22T23:15:18.760 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-22T23:15:18.764 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-22T23:15:18.823 INFO:teuthology.orchestra.run.trial040.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-22T23:15:18.823 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:18.824 INFO:teuthology.orchestra.run.trial040.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-22T23:15:18.843 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:18.843 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:18.843 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-22T23:15:18.843 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:18.843 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:18.843 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:18.846 INFO:teuthology.orchestra.run.trial040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-22T23:15:18.860 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:18.860 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:18.861 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:19.005 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:47 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:19.070 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:19.076 INFO:teuthology.orchestra.run.trial006.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-22T23:15:19.080 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-22T23:15:19.088 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:19.088 INFO:teuthology.orchestra.run.trial006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-22T23:15:19.089 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:19.089 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout:Installing dependencies: 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout:Installing weak dependencies: 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:19.090 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:19.091 INFO:teuthology.orchestra.run.trial080.stdout:Install 3 Packages 2026-02-22T23:15:19.091 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:19.091 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 107 M 2026-02-22T23:15:19.091 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 654 M 2026-02-22T23:15:19.091 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:19.091 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:15:19.171 INFO:teuthology.orchestra.run.trial040.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-22T23:15:19.244 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-22T23:15:19.485 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:19.485 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-22T23:15:19.485 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-22T23:15:19.485 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-22T23:15:19.486 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-22T23:15:19.487 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-22T23:15:19.488 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-22T23:15:19.489 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-22T23:15:19.490 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-22T23:15:19.491 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-22T23:15:19.492 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-22T23:15:19.492 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-22T23:15:19.550 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-22T23:15:19.550 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:19.550 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:19.550 INFO:teuthology.orchestra.run.trial189.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.551 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-22T23:15:19.552 INFO:teuthology.orchestra.run.trial189.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: lua-5.4.4-4.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-22T23:15:19.553 INFO:teuthology.orchestra.run.trial189.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-22T23:15:19.554 INFO:teuthology.orchestra.run.trial189.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-22T23:15:19.555 INFO:teuthology.orchestra.run.trial189.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-ply-3.11-14.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-22T23:15:19.556 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-22T23:15:19.557 INFO:teuthology.orchestra.run.trial189.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: unzip-6.0-59.el9.x86_64 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: zip-3.0-35.el9.x86_64 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:19.558 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:19.607 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-base 2026-02-22T23:15:19.914 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-22T23:15:19.922 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:35 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:19.931 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-22T23:15:19.941 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-22T23:15:19.948 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-22T23:15:19.961 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-22T23:15:19.977 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-22T23:15:19.985 INFO:teuthology.orchestra.run.trial080.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 12 MB/s | 11 MB 00:00 2026-02-22T23:15:19.986 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:20.005 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:20.005 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:20.005 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:20.027 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install cephadm 2026-02-22T23:15:20.052 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-22T23:15:20.061 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-22T23:15:20.084 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-22T23:15:20.114 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-22T23:15:20.165 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-22T23:15:20.172 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-22T23:15:20.175 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-22T23:15:20.176 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-22T23:15:20.194 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:20.195 INFO:teuthology.orchestra.run.trial040.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-22T23:15:20.195 INFO:teuthology.orchestra.run.trial040.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-22T23:15:20.195 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:20.202 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:20.228 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:20.228 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-22T23:15:20.228 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:20.262 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-22T23:15:20.327 INFO:teuthology.orchestra.run.trial040.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-22T23:15:20.329 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:20.342 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:20.343 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:20.343 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:20.343 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-22T23:15:20.343 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:20.407 INFO:teuthology.orchestra.run.trial189.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:20.425 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:20.425 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:20.426 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:20.446 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-immutable-object-cache 2026-02-22T23:15:20.754 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:20.830 INFO:teuthology.orchestra.run.trial080.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 14 MB/s | 24 MB 00:01 2026-02-22T23:15:20.836 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:20.837 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:20.837 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:15:20.837 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:20.837 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 145 k 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 431 k 2026-02-22T23:15:20.838 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:21.041 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:21.044 INFO:teuthology.orchestra.run.trial189.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 703 kB/s | 145 kB 00:00 2026-02-22T23:15:21.046 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:21.046 INFO:teuthology.orchestra.run.trial189.stdout:Total 694 kB/s | 145 kB 00:00 2026-02-22T23:15:21.047 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:21.051 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:21.051 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:21.067 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:21.067 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:21.067 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-22T23:15:21.067 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:21.067 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:21.068 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:21.082 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:21.083 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:21.110 INFO:teuthology.orchestra.run.trial080.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 36 MB/s | 72 MB 00:02 2026-02-22T23:15:21.112 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:21.112 INFO:teuthology.orchestra.run.trial080.stdout:Total 53 MB/s | 107 MB 00:02 2026-02-22T23:15:21.113 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:21.118 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:21.118 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:21.128 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:21.130 INFO:teuthology.orchestra.run.trial040.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:21.134 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-22T23:15:21.138 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:21.139 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:21.154 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-22T23:15:21.155 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:21.160 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:21.161 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:21.161 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-22T23:15:21.161 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:21.334 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-22T23:15:21.352 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:15:21.451 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:21.451 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:21.451 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:21.451 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:21.451 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:21.451 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:21.478 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr 2026-02-22T23:15:21.498 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:21.498 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:21.587 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-22T23:15:21.587 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-22T23:15:21.587 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-22T23:15:21.587 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-22T23:15:21.587 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:21.641 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:21.648 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:21.652 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:21.675 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph 2026-02-22T23:15:21.792 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:37 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:21.856 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:21.875 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:21.876 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:21.876 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:21.897 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-dashboard 2026-02-22T23:15:21.982 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:17 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:22.070 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:22.072 INFO:teuthology.orchestra.run.trial006.stdout:====================================================================================== 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repository Size 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout:====================================================================================== 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:15:22.073 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-22T23:15:22.074 INFO:teuthology.orchestra.run.trial006.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-22T23:15:22.075 INFO:teuthology.orchestra.run.trial006.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-22T23:15:22.076 INFO:teuthology.orchestra.run.trial006.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-22T23:15:22.077 INFO:teuthology.orchestra.run.trial006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-22T23:15:22.078 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-22T23:15:22.079 INFO:teuthology.orchestra.run.trial006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-22T23:15:22.080 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout:Installing weak dependencies: 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:22.081 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:22.082 INFO:teuthology.orchestra.run.trial006.stdout:====================================================================================== 2026-02-22T23:15:22.082 INFO:teuthology.orchestra.run.trial006.stdout:Install 99 Packages 2026-02-22T23:15:22.082 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:22.083 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 95 M 2026-02-22T23:15:22.083 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 520 M 2026-02-22T23:15:22.083 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:22.104 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:22.105 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:22.118 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:22.119 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:22.176 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:22.211 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:22.226 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-22T23:15:22.227 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:22.249 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:22.249 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:22.250 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-22T23:15:22.250 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:22.250 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:22.250 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:22.262 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:22.276 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:22.294 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:22.294 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:22.295 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:22.315 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-22T23:15:22.479 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:22.481 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:22.503 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:22.503 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:22.503 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-22T23:15:22.504 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:22.504 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:22.504 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:22.608 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:22.630 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:22.630 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:22.630 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-22T23:15:22.630 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:22.630 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:22.630 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:22.631 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:22.648 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:22.664 INFO:teuthology.orchestra.run.trial006.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 110 kB/s | 6.5 kB 00:00 2026-02-22T23:15:22.695 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:22.713 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:22.713 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:22.714 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:22.736 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-rook 2026-02-22T23:15:22.742 INFO:teuthology.orchestra.run.trial006.stdout:(2/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 7.9 MB/s | 1.1 MB 00:00 2026-02-22T23:15:22.762 INFO:teuthology.orchestra.run.trial006.stdout:(3/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 15 MB/s | 2.4 MB 00:00 2026-02-22T23:15:22.786 INFO:teuthology.orchestra.run.trial006.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 1.3 MB/s | 31 kB 00:00 2026-02-22T23:15:22.817 INFO:teuthology.orchestra.run.trial006.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 4.8 MB/s | 150 kB 00:00 2026-02-22T23:15:22.959 INFO:teuthology.orchestra.run.trial006.stdout:(6/99): ceph-mgr-dashboard-19.2.3-689.gf90f4eae 27 MB/s | 3.8 MB 00:00 2026-02-22T23:15:23.045 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:23.048 INFO:teuthology.orchestra.run.trial006.stdout:(7/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 12 MB/s | 4.7 MB 00:00 2026-02-22T23:15:23.109 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:23.128 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:23.129 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:23.129 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:23.134 INFO:teuthology.orchestra.run.trial006.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 257 kB/s | 22 kB 00:00 2026-02-22T23:15:23.151 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-cephadm 2026-02-22T23:15:23.166 INFO:teuthology.orchestra.run.trial006.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 7.6 MB/s | 253 kB 00:00 2026-02-22T23:15:23.196 INFO:teuthology.orchestra.run.trial006.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 1.6 MB/s | 49 kB 00:00 2026-02-22T23:15:23.225 INFO:teuthology.orchestra.run.trial006.stdout:(11/99): ceph-mgr-diskprediction-local-19.2.3-6 28 MB/s | 7.4 MB 00:00 2026-02-22T23:15:23.225 INFO:teuthology.orchestra.run.trial006.stdout:(12/99): ceph-prometheus-alerts-19.2.3-689.gf90 568 kB/s | 17 kB 00:00 2026-02-22T23:15:23.237 INFO:teuthology.orchestra.run.trial006.stdout:(13/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 23 MB/s | 299 kB 00:00 2026-02-22T23:15:23.258 INFO:teuthology.orchestra.run.trial006.stdout:(14/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 23 MB/s | 769 kB 00:00 2026-02-22T23:15:23.310 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:23.310 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-22T23:15:23.310 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-22T23:15:23.310 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-22T23:15:23.310 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-22T23:15:23.311 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-22T23:15:23.312 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-22T23:15:23.313 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-22T23:15:23.314 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-22T23:15:23.315 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-22T23:15:23.316 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-22T23:15:23.317 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-22T23:15:23.318 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-22T23:15:23.318 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-22T23:15:23.318 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-22T23:15:23.373 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-22T23:15:23.373 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:23.373 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:23.373 INFO:teuthology.orchestra.run.trial040.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-22T23:15:23.373 INFO:teuthology.orchestra.run.trial040.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:23.374 INFO:teuthology.orchestra.run.trial040.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-22T23:15:23.375 INFO:teuthology.orchestra.run.trial040.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: lua-5.4.4-4.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-22T23:15:23.376 INFO:teuthology.orchestra.run.trial040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-22T23:15:23.377 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-22T23:15:23.378 INFO:teuthology.orchestra.run.trial040.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-ply-3.11-14.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-22T23:15:23.379 INFO:teuthology.orchestra.run.trial040.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-22T23:15:23.380 INFO:teuthology.orchestra.run.trial040.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: unzip-6.0-59.el9.x86_64 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-22T23:15:23.381 INFO:teuthology.orchestra.run.trial040.stdout: zip-3.0-35.el9.x86_64 2026-02-22T23:15:23.382 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:23.382 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:23.430 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-base 2026-02-22T23:15:23.442 INFO:teuthology.orchestra.run.trial006.stdout:(15/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_ 24 MB/s | 17 MB 00:00 2026-02-22T23:15:23.464 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:23.528 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:23.546 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:23.547 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:23.547 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:23.568 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-fuse 2026-02-22T23:15:23.677 INFO:teuthology.orchestra.run.trial006.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 96 kB/s | 40 kB 00:00 2026-02-22T23:15:23.740 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:30 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:23.805 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:23.823 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:23.824 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:23.825 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:23.846 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install cephadm 2026-02-22T23:15:23.880 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:23.925 INFO:teuthology.orchestra.run.trial006.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 149 kB/s | 72 kB 00:00 2026-02-22T23:15:23.952 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:15:23.963 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:23.963 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:23.963 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:23.963 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:23.963 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:23.963 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-22T23:15:23.963 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:23.964 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:23.964 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:23.964 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:23.964 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:23.964 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 1.2 M 2026-02-22T23:15:23.964 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 3.6 M 2026-02-22T23:15:23.964 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:24.046 INFO:teuthology.orchestra.run.trial006.stdout:(18/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 434 kB/s | 351 kB 00:00 2026-02-22T23:15:24.085 INFO:teuthology.orchestra.run.trial189.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 9.5 MB/s | 1.2 MB 00:00 2026-02-22T23:15:24.085 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:24.085 INFO:teuthology.orchestra.run.trial189.stdout:Total 9.5 MB/s | 1.2 MB 00:00 2026-02-22T23:15:24.085 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:24.089 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:24.089 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:24.126 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:24.126 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:24.159 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:31 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:24.165 INFO:teuthology.orchestra.run.trial006.stdout:(19/99): libquadmath-11.5.0-14.el9.x86_64.rpm 768 kB/s | 184 kB 00:00 2026-02-22T23:15:24.225 INFO:teuthology.orchestra.run.trial040.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:24.229 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:24.243 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:24.244 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:24.244 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:24.246 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:24.266 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-immutable-object-cache 2026-02-22T23:15:24.366 INFO:teuthology.orchestra.run.trial006.stdout:(20/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 794 kB/s | 253 kB 00:00 2026-02-22T23:15:24.463 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:24.521 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:24.521 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:24.521 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:24.521 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:24.521 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:24.521 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:24.538 INFO:teuthology.orchestra.run.trial006.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 616 kB/s | 106 kB 00:00 2026-02-22T23:15:24.551 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-volume 2026-02-22T23:15:24.577 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:31 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:24.627 INFO:teuthology.orchestra.run.trial006.stdout:(22/99): libgfortran-11.5.0-14.el9.x86_64.rpm 836 kB/s | 794 kB 00:00 2026-02-22T23:15:24.660 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:24.661 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:24.661 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:15:24.661 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:24.661 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 145 k 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 431 k 2026-02-22T23:15:24.662 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:24.717 INFO:teuthology.orchestra.run.trial006.stdout:(23/99): python3-pycparser-2.20-6.el9.noarch.rp 757 kB/s | 135 kB 00:00 2026-02-22T23:15:24.763 INFO:teuthology.orchestra.run.trial040.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 1.4 MB/s | 145 kB 00:00 2026-02-22T23:15:24.763 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:24.763 INFO:teuthology.orchestra.run.trial040.stdout:Total 1.4 MB/s | 145 kB 00:00 2026-02-22T23:15:24.763 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:24.766 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:24.766 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:24.798 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:24.798 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:24.853 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:24.854 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:24.866 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:40 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:24.869 INFO:teuthology.orchestra.run.trial006.stdout:(24/99): python3-pyparsing-2.4.7-9.el9.noarch.r 622 kB/s | 150 kB 00:00 2026-02-22T23:15:24.877 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:24.877 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:24.878 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-22T23:15:24.878 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:24.884 INFO:teuthology.orchestra.run.trial006.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 759 kB/s | 126 kB 00:00 2026-02-22T23:15:24.931 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:24.950 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:24.950 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:24.950 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:24.973 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados-devel 2026-02-22T23:15:25.124 INFO:teuthology.orchestra.run.trial006.stdout:(26/99): unzip-6.0-59.el9.x86_64.rpm 758 kB/s | 182 kB 00:00 2026-02-22T23:15:25.182 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:25.182 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:25.182 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:25.182 INFO:teuthology.orchestra.run.trial040.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:25.182 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:25.182 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:25.185 INFO:teuthology.orchestra.run.trial006.stdout:(27/99): python3-urllib3-1.26.5-7.el9.noarch.rp 689 kB/s | 218 kB 00:00 2026-02-22T23:15:25.211 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr 2026-02-22T23:15:25.222 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-22T23:15:25.277 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:41 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:25.362 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:25.362 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:25.362 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:15:25.362 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:25.362 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:25.362 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-22T23:15:25.362 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:25.363 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:25.363 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:25.363 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:25.363 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:25.363 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 126 k 2026-02-22T23:15:25.363 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 456 k 2026-02-22T23:15:25.363 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:25.443 INFO:teuthology.orchestra.run.trial006.stdout:(28/99): zip-3.0-35.el9.x86_64.rpm 833 kB/s | 266 kB 00:00 2026-02-22T23:15:25.450 INFO:teuthology.orchestra.run.trial189.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 1.4 MB/s | 126 kB 00:00 2026-02-22T23:15:25.451 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:25.451 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-22T23:15:25.451 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:25.453 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:25.453 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:25.455 INFO:teuthology.orchestra.run.trial006.stdout:(29/99): flexiblas-3.0.4-9.el9.x86_64.rpm 110 kB/s | 30 kB 00:00 2026-02-22T23:15:25.486 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:25.486 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:25.519 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:32 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:25.540 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:25.557 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:25.559 INFO:teuthology.orchestra.run.trial006.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 143 kB/s | 15 kB 00:00 2026-02-22T23:15:25.583 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:25.602 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:25.603 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:25.603 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:25.624 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-dashboard 2026-02-22T23:15:25.636 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:25.663 INFO:teuthology.orchestra.run.trial006.stdout:(31/99): python3-cryptography-36.0.1-5.el9.x86_ 852 kB/s | 1.2 MB 00:01 2026-02-22T23:15:25.695 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:25.695 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:25.695 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:25.695 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:25.696 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:25.696 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:25.723 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install libcephfs2 2026-02-22T23:15:25.902 INFO:teuthology.orchestra.run.trial006.stdout:(32/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 720 kB/s | 246 kB 00:00 2026-02-22T23:15:25.932 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:32 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:25.991 INFO:teuthology.orchestra.run.trial006.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 476 kB/s | 42 kB 00:00 2026-02-22T23:15:25.999 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:26.017 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:26.017 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:26.017 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:26.039 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-22T23:15:26.042 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:26.106 INFO:teuthology.orchestra.run.trial189.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:26.125 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:26.125 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:26.125 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:26.129 INFO:teuthology.orchestra.run.trial006.stdout:(34/99): lua-5.4.4-4.el9.x86_64.rpm 404 kB/s | 188 kB 00:00 2026-02-22T23:15:26.148 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install libcephfs-devel 2026-02-22T23:15:26.220 INFO:teuthology.orchestra.run.trial006.stdout:(35/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 3.8 MB/s | 3.0 MB 00:00 2026-02-22T23:15:26.349 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:33 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:26.416 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:26.431 INFO:teuthology.orchestra.run.trial006.stdout:(36/99): protobuf-3.14.0-17.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2026-02-22T23:15:26.434 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:26.434 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:26.435 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:26.440 INFO:teuthology.orchestra.run.trial006.stdout:(37/99): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2026-02-22T23:15:26.450 INFO:teuthology.orchestra.run.trial006.stdout:(38/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 11 MB/s | 5.3 MB 00:00 2026-02-22T23:15:26.456 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-rook 2026-02-22T23:15:26.462 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:26.521 INFO:teuthology.orchestra.run.trial006.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 3.0 MB/s | 249 kB 00:00 2026-02-22T23:15:26.522 INFO:teuthology.orchestra.run.trial006.stdout:(40/99): python3-devel-3.9.25-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2026-02-22T23:15:26.533 INFO:teuthology.orchestra.run.trial006.stdout:(41/99): python3-libstoragemgmt-1.10.1-1.el9.x8 2.1 MB/s | 177 kB 00:00 2026-02-22T23:15:26.545 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:26.546 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:26.546 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:15:26.546 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:26.546 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:26.546 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-22T23:15:26.546 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:26.546 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:26.547 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:26.547 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:26.547 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:26.547 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 34 k 2026-02-22T23:15:26.547 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 153 k 2026-02-22T23:15:26.547 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:26.607 INFO:teuthology.orchestra.run.trial006.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2026-02-22T23:15:26.611 INFO:teuthology.orchestra.run.trial189.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 516 kB/s | 34 kB 00:00 2026-02-22T23:15:26.611 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:26.611 INFO:teuthology.orchestra.run.trial006.stdout:(43/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2026-02-22T23:15:26.611 INFO:teuthology.orchestra.run.trial189.stdout:Total 511 kB/s | 34 kB 00:00 2026-02-22T23:15:26.611 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:26.614 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:26.614 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:26.615 INFO:teuthology.orchestra.run.trial006.stdout:(44/99): python3-markupsafe-1.1.1-12.el9.x86_64 426 kB/s | 35 kB 00:00 2026-02-22T23:15:26.616 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:26.616 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:26.637 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:26.653 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:26.693 INFO:teuthology.orchestra.run.trial006.stdout:(45/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 5.3 MB/s | 442 kB 00:00 2026-02-22T23:15:26.695 INFO:teuthology.orchestra.run.trial006.stdout:(46/99): python3-packaging-20.9-5.el9.noarch.rp 961 kB/s | 77 kB 00:00 2026-02-22T23:15:26.713 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:26.767 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:33 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:26.772 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:26.773 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:26.773 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:26.773 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:26.773 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:26.773 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:26.774 INFO:teuthology.orchestra.run.trial006.stdout:(47/99): python3-protobuf-3.14.0-17.el9.noarch. 3.2 MB/s | 267 kB 00:00 2026-02-22T23:15:26.777 INFO:teuthology.orchestra.run.trial006.stdout:(48/99): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2026-02-22T23:15:26.802 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados2 2026-02-22T23:15:26.833 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:26.851 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:26.852 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:26.852 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:26.856 INFO:teuthology.orchestra.run.trial006.stdout:(49/99): python3-pyasn1-modules-0.4.8-6.el9.noa 3.3 MB/s | 279 kB 00:00 2026-02-22T23:15:26.857 INFO:teuthology.orchestra.run.trial006.stdout:(50/99): python3-requests-oauthlib-1.3.0-12.el9 664 kB/s | 54 kB 00:00 2026-02-22T23:15:26.873 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-cephadm 2026-02-22T23:15:26.915 INFO:teuthology.orchestra.run.trial006.stdout:(51/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 20 MB/s | 6.1 MB 00:00 2026-02-22T23:15:26.937 INFO:teuthology.orchestra.run.trial006.stdout:(52/99): python3-toml-0.10.2-6.el9.noarch.rpm 522 kB/s | 42 kB 00:00 2026-02-22T23:15:26.996 INFO:teuthology.orchestra.run.trial006.stdout:(53/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 2.3 MB/s | 189 kB 00:00 2026-02-22T23:15:27.017 INFO:teuthology.orchestra.run.trial006.stdout:(54/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 1.1 MB/s | 90 kB 00:00 2026-02-22T23:15:27.113 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:27.178 INFO:teuthology.orchestra.run.trial189.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:27.182 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:34 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:27.196 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:27.197 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:27.197 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:27.219 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librbd1 2026-02-22T23:15:27.246 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:27.264 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:27.265 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:27.265 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:27.286 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-fuse 2026-02-22T23:15:27.449 INFO:teuthology.orchestra.run.trial006.stdout:(55/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 33 MB/s | 19 MB 00:00 2026-02-22T23:15:27.528 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:27.537 INFO:teuthology.orchestra.run.trial006.stdout:(56/99): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2026-02-22T23:15:27.593 INFO:teuthology.orchestra.run.trial189.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:27.595 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:34 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:27.612 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:27.612 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:27.612 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:27.634 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rados 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:27.678 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:27.679 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:27.679 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:27.679 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 1.2 M 2026-02-22T23:15:27.679 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 3.6 M 2026-02-22T23:15:27.679 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:27.709 INFO:teuthology.orchestra.run.trial006.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 113 kB/s | 19 kB 00:00 2026-02-22T23:15:27.802 INFO:teuthology.orchestra.run.trial040.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 9.4 MB/s | 1.2 MB 00:00 2026-02-22T23:15:27.803 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:27.804 INFO:teuthology.orchestra.run.trial040.stdout:Total 9.2 MB/s | 1.2 MB 00:00 2026-02-22T23:15:27.804 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:27.806 INFO:teuthology.orchestra.run.trial006.stdout:(58/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.5 MB/s | 551 kB 00:00 2026-02-22T23:15:27.807 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:27.807 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:27.844 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:27.844 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:27.854 INFO:teuthology.orchestra.run.trial006.stdout:(59/99): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2026-02-22T23:15:27.874 INFO:teuthology.orchestra.run.trial006.stdout:(60/99): python3-asyncssh-2.13.2-5.el9.noarch.r 7.8 MB/s | 548 kB 00:00 2026-02-22T23:15:27.901 INFO:teuthology.orchestra.run.trial006.stdout:(61/99): python3-autocommand-2.2.2-8.el9.noarch 626 kB/s | 29 kB 00:00 2026-02-22T23:15:27.917 INFO:teuthology.orchestra.run.trial006.stdout:(62/99): python3-backports-tarfile-1.2.0-1.el9. 1.4 MB/s | 60 kB 00:00 2026-02-22T23:15:27.919 INFO:teuthology.orchestra.run.trial006.stdout:(63/99): protobuf-compiler-3.14.0-17.el9.x86_64 956 kB/s | 862 kB 00:00 2026-02-22T23:15:27.947 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:27.947 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:27.950 INFO:teuthology.orchestra.run.trial006.stdout:(64/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 888 kB/s | 43 kB 00:00 2026-02-22T23:15:27.959 INFO:teuthology.orchestra.run.trial006.stdout:(65/99): python3-cachetools-4.2.4-1.el9.noarch. 779 kB/s | 32 kB 00:00 2026-02-22T23:15:27.964 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:28.012 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:28.015 INFO:teuthology.orchestra.run.trial006.stdout:(66/99): python3-cherrypy-18.6.1-2.el9.noarch.r 6.2 MB/s | 358 kB 00:00 2026-02-22T23:15:28.022 INFO:teuthology.orchestra.run.trial006.stdout:(67/99): python3-cheroot-10.0.1-4.el9.noarch.rp 2.4 MB/s | 173 kB 00:00 2026-02-22T23:15:28.030 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:28.030 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:28.031 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:28.053 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rgw 2026-02-22T23:15:28.067 INFO:teuthology.orchestra.run.trial006.stdout:(68/99): python3-google-auth-2.45.0-1.el9.noarc 4.8 MB/s | 254 kB 00:00 2026-02-22T23:15:28.086 INFO:teuthology.orchestra.run.trial006.stdout:(69/99): python3-certifi-2023.05.07-4.el9.noarc 84 kB/s | 14 kB 00:00 2026-02-22T23:15:28.115 INFO:teuthology.orchestra.run.trial006.stdout:(70/99): python3-grpcio-tools-1.46.7-10.el9.x86 3.0 MB/s | 144 kB 00:00 2026-02-22T23:15:28.157 INFO:teuthology.orchestra.run.trial006.stdout:(71/99): python3-isodate-0.6.1-3.el9.noarch.rpm 1.3 MB/s | 56 kB 00:00 2026-02-22T23:15:28.199 INFO:teuthology.orchestra.run.trial006.stdout:(72/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 256 kB/s | 11 kB 00:00 2026-02-22T23:15:28.208 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:28.210 INFO:teuthology.orchestra.run.trial006.stdout:(73/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 11 MB/s | 2.0 MB 00:00 2026-02-22T23:15:28.231 INFO:teuthology.orchestra.run.trial006.stdout:(74/99): python3-influxdb-5.3.1-1.el9.noarch.rp 958 kB/s | 139 kB 00:00 2026-02-22T23:15:28.241 INFO:teuthology.orchestra.run.trial006.stdout:(75/99): python3-jaraco-classes-3.2.1-5.el9.noa 419 kB/s | 18 kB 00:00 2026-02-22T23:15:28.252 INFO:teuthology.orchestra.run.trial006.stdout:(76/99): python3-jaraco-collections-3.0.0-8.el9 550 kB/s | 23 kB 00:00 2026-02-22T23:15:28.265 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:28.265 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:28.265 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:28.265 INFO:teuthology.orchestra.run.trial040.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:28.265 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:28.266 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:28.276 INFO:teuthology.orchestra.run.trial006.stdout:(77/99): python3-jaraco-context-6.0.1-3.el9.noa 437 kB/s | 20 kB 00:00 2026-02-22T23:15:28.282 INFO:teuthology.orchestra.run.trial006.stdout:(78/99): python3-jaraco-functools-3.5.0-2.el9.n 471 kB/s | 19 kB 00:00 2026-02-22T23:15:28.294 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-volume 2026-02-22T23:15:28.295 INFO:teuthology.orchestra.run.trial006.stdout:(79/99): python3-jaraco-text-4.0.0-2.el9.noarch 631 kB/s | 26 kB 00:00 2026-02-22T23:15:28.324 INFO:teuthology.orchestra.run.trial006.stdout:(80/99): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2026-02-22T23:15:28.337 INFO:teuthology.orchestra.run.trial006.stdout:(81/99): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2026-02-22T23:15:28.365 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:44 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:28.368 INFO:teuthology.orchestra.run.trial006.stdout:(82/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.9 MB/s | 86 kB 00:00 2026-02-22T23:15:28.378 INFO:teuthology.orchestra.run.trial006.stdout:(83/99): python3-natsort-7.1.1-5.el9.noarch.rpm 1.4 MB/s | 58 kB 00:00 2026-02-22T23:15:28.420 INFO:teuthology.orchestra.run.trial006.stdout:(84/99): python3-portend-3.1.0-2.el9.noarch.rpm 396 kB/s | 16 kB 00:00 2026-02-22T23:15:28.420 INFO:teuthology.orchestra.run.trial006.stdout:(85/99): python3-pecan-1.4.2-3.el9.noarch.rpm 5.0 MB/s | 272 kB 00:00 2026-02-22T23:15:28.430 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:28.444 INFO:teuthology.orchestra.run.trial006.stdout:(86/99): python3-kubernetes-26.1.0-3.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2026-02-22T23:15:28.449 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:28.449 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:28.449 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:28.462 INFO:teuthology.orchestra.run.trial006.stdout:(87/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2026-02-22T23:15:28.463 INFO:teuthology.orchestra.run.trial006.stdout:(88/99): python3-repoze-lru-0.7-16.el9.noarch.r 729 kB/s | 31 kB 00:00 2026-02-22T23:15:28.471 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-cephfs 2026-02-22T23:15:28.490 INFO:teuthology.orchestra.run.trial006.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 4.0 MB/s | 188 kB 00:00 2026-02-22T23:15:28.505 INFO:teuthology.orchestra.run.trial006.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2026-02-22T23:15:28.506 INFO:teuthology.orchestra.run.trial006.stdout:(91/99): python3-saml-1.16.0-1.el9.noarch.rpm 2.9 MB/s | 125 kB 00:00 2026-02-22T23:15:28.533 INFO:teuthology.orchestra.run.trial006.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 842 kB/s | 36 kB 00:00 2026-02-22T23:15:28.549 INFO:teuthology.orchestra.run.trial006.stdout:(93/99): python3-typing-extensions-4.15.0-1.el9 1.9 MB/s | 86 kB 00:00 2026-02-22T23:15:28.550 INFO:teuthology.orchestra.run.trial006.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 5.1 MB/s | 230 kB 00:00 2026-02-22T23:15:28.576 INFO:teuthology.orchestra.run.trial006.stdout:(95/99): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2026-02-22T23:15:28.592 INFO:teuthology.orchestra.run.trial006.stdout:(96/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.1 MB/s | 48 kB 00:00 2026-02-22T23:15:28.608 INFO:teuthology.orchestra.run.trial006.stdout:(97/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 7.1 MB/s | 427 kB 00:00 2026-02-22T23:15:28.610 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:35 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:28.619 INFO:teuthology.orchestra.run.trial006.stdout:(98/99): python3-xmltodict-0.12.0-15.el9.noarch 515 kB/s | 22 kB 00:00 2026-02-22T23:15:28.635 INFO:teuthology.orchestra.run.trial006.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 470 kB/s | 20 kB 00:00 2026-02-22T23:15:28.638 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:28.638 INFO:teuthology.orchestra.run.trial006.stdout:Total 15 MB/s | 95 MB 00:06 2026-02-22T23:15:28.675 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:28.693 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:28.694 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:28.694 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:28.715 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librados-devel 2026-02-22T23:15:28.782 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:44 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:28.846 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:28.865 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:28.865 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:28.866 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:28.888 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rbd 2026-02-22T23:15:28.904 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:15:28.924 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:15:29.025 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:29.080 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:29.109 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:29.109 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 126 k 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 456 k 2026-02-22T23:15:29.110 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:29.118 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:29.118 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:29.196 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:45 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:29.198 INFO:teuthology.orchestra.run.trial040.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 1.4 MB/s | 126 kB 00:00 2026-02-22T23:15:29.199 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:29.199 INFO:teuthology.orchestra.run.trial040.stdout:Total 1.4 MB/s | 126 kB 00:00 2026-02-22T23:15:29.199 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:29.202 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:29.202 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:29.235 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:29.235 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:29.261 INFO:teuthology.orchestra.run.trial189.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:29.279 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:29.280 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:15:29.280 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:29.290 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:29.301 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-fuse 2026-02-22T23:15:29.307 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:29.389 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:29.446 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:29.446 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:29.446 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:29.446 INFO:teuthology.orchestra.run.trial040.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:29.447 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:29.447 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:29.447 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:15:29.448 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-22T23:15:29.448 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:15:29.474 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install libcephfs2 2026-02-22T23:15:29.506 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-22T23:15:29.507 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:29.507 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:15:29.507 INFO:teuthology.orchestra.run.trial080.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:29.507 INFO:teuthology.orchestra.run.trial080.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:29.507 INFO:teuthology.orchestra.run.trial080.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:29.507 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:29.507 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:15:29.533 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:29.533 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:29.547 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-common-debuginfo 2026-02-22T23:15:29.615 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:45 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:29.698 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:29.699 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:29.700 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:29.700 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:29.700 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 85 k 2026-02-22T23:15:29.700 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 227 k 2026-02-22T23:15:29.700 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:29.776 INFO:teuthology.orchestra.run.trial189.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-22T23:15:29.776 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:29.776 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-22T23:15:29.777 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:29.780 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:29.780 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:29.785 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:29.811 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:29.811 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:29.851 INFO:teuthology.orchestra.run.trial040.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:29.857 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:57 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:15:29.862 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:29.870 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:29.871 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:29.872 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:29.879 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:29.893 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install libcephfs-devel 2026-02-22T23:15:29.942 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:15:29.943 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:29.943 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:15:29.943 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:29.943 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:15:29.943 INFO:teuthology.orchestra.run.trial080.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 474 M 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 2.0 G 2026-02-22T23:15:29.944 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:15:29.963 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:30.001 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:30.003 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-22T23:15:30.021 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:30.022 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:30.022 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:30.022 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:30.022 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:30.022 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:30.034 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-22T23:15:30.039 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-22T23:15:30.049 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-mirror 2026-02-22T23:15:30.061 INFO:teuthology.orchestra.run.trial006.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-22T23:15:30.067 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 5/99 2026-02-22T23:15:30.100 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-22T23:15:30.110 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-22T23:15:30.115 INFO:teuthology.orchestra.run.trial006.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-22T23:15:30.117 INFO:teuthology.orchestra.run.trial006.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-22T23:15:30.122 INFO:teuthology.orchestra.run.trial006.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-22T23:15:30.145 INFO:teuthology.orchestra.run.trial006.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-22T23:15:30.157 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-22T23:15:30.160 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-22T23:15:30.166 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-22T23:15:30.168 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-22T23:15:30.179 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-22T23:15:30.186 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-22T23:15:30.207 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:37 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:30.210 INFO:teuthology.orchestra.run.trial006.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-22T23:15:30.213 INFO:teuthology.orchestra.run.trial006.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-22T23:15:30.220 INFO:teuthology.orchestra.run.trial006.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-22T23:15:30.244 INFO:teuthology.orchestra.run.trial006.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-22T23:15:30.290 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:30.290 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:30.290 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:15:30.290 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:30.290 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:30.290 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-22T23:15:30.290 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:30.291 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:30.291 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:30.291 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:30.291 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:30.291 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 34 k 2026-02-22T23:15:30.291 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 153 k 2026-02-22T23:15:30.291 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:30.295 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-22T23:15:30.309 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 23/99 2026-02-22T23:15:30.311 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-22T23:15:30.316 INFO:teuthology.orchestra.run.trial006.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-22T23:15:30.321 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-22T23:15:30.326 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-22T23:15:30.329 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-22T23:15:30.343 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-22T23:15:30.364 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-22T23:15:30.368 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-22T23:15:30.371 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:46 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:30.373 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-22T23:15:30.385 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-22T23:15:30.394 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-22T23:15:30.400 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-22T23:15:30.420 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-22T23:15:30.429 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-22T23:15:30.454 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 3.1 M 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 13 M 2026-02-22T23:15:30.455 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:30.483 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-22T23:15:30.496 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-22T23:15:30.511 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-22T23:15:30.515 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-22T23:15:30.523 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-22T23:15:30.563 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-22T23:15:30.572 INFO:teuthology.orchestra.run.trial040.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 120 kB/s | 34 kB 00:00 2026-02-22T23:15:30.572 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:30.572 INFO:teuthology.orchestra.run.trial040.stdout:Total 119 kB/s | 34 kB 00:00 2026-02-22T23:15:30.572 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:30.574 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:30.574 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:30.577 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:30.577 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:30.599 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:30.616 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:30.670 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:30.726 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:30.726 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:30.727 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:30.727 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:30.727 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:30.727 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:30.754 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librados2 2026-02-22T23:15:30.890 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-22T23:15:30.900 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-22T23:15:30.904 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-22T23:15:30.910 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-22T23:15:30.912 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-22T23:15:30.940 INFO:teuthology.orchestra.run.trial006.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-22T23:15:30.983 INFO:teuthology.orchestra.run.trial006.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-22T23:15:30.990 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-22T23:15:30.993 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-22T23:15:30.999 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-22T23:15:31.002 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-22T23:15:31.009 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-22T23:15:31.013 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-22T23:15:31.041 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-22T23:15:31.051 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-22T23:15:31.067 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:31.085 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-22T23:15:31.133 INFO:teuthology.orchestra.run.trial040.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:31.152 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:31.152 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:31.153 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:31.174 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librbd1 2026-02-22T23:15:31.310 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-22T23:15:31.334 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-22T23:15:31.338 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-22T23:15:31.396 INFO:teuthology.orchestra.run.trial006.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-22T23:15:31.398 INFO:teuthology.orchestra.run.trial006.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-22T23:15:31.419 INFO:teuthology.orchestra.run.trial189.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 3.2 MB/s | 3.1 MB 00:00 2026-02-22T23:15:31.419 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:31.419 INFO:teuthology.orchestra.run.trial189.stdout:Total 3.2 MB/s | 3.1 MB 00:00 2026-02-22T23:15:31.419 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:31.420 INFO:teuthology.orchestra.run.trial006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-22T23:15:31.423 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:31.423 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:31.464 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:31.464 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:31.488 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:31.552 INFO:teuthology.orchestra.run.trial040.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:31.571 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:31.572 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:31.572 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:31.593 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rados 2026-02-22T23:15:31.658 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:31.659 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:31.683 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:31.684 INFO:teuthology.orchestra.run.trial189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:31.684 INFO:teuthology.orchestra.run.trial189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-22T23:15:31.684 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:31.684 INFO:teuthology.orchestra.run.trial189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:31.684 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:31.744 INFO:teuthology.orchestra.run.trial006.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-22T23:15:31.817 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-22T23:15:31.909 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:31.976 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:31.986 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:31.987 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:31.987 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:31.987 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:31.987 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:31.987 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:31.994 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:31.995 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:31.995 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:32.016 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-nbd 2026-02-22T23:15:32.018 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rgw 2026-02-22T23:15:32.322 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:32.332 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:48 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:32.389 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:32.407 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:32.408 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:32.408 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:32.415 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:32.416 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:32.416 INFO:teuthology.orchestra.run.trial189.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:32.416 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:32.416 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:32.416 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 170 k 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 490 k 2026-02-22T23:15:32.417 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:32.429 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-cephfs 2026-02-22T23:15:32.487 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-22T23:15:32.504 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-22T23:15:32.514 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-22T23:15:32.521 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-22T23:15:32.534 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-22T23:15:32.550 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-22T23:15:32.626 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-22T23:15:32.634 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-22T23:15:32.658 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-22T23:15:32.687 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-22T23:15:32.732 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:32.738 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-22T23:15:32.744 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-22T23:15:32.748 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-22T23:15:32.749 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-22T23:15:32.767 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:32.767 INFO:teuthology.orchestra.run.trial006.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-22T23:15:32.767 INFO:teuthology.orchestra.run.trial006.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-22T23:15:32.767 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:32.775 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:32.799 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-22T23:15:32.799 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-22T23:15:32.800 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:32.800 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:32.817 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:32.818 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:32.818 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:32.833 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-22T23:15:32.838 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rbd 2026-02-22T23:15:32.841 INFO:teuthology.orchestra.run.trial189.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 402 kB/s | 170 kB 00:00 2026-02-22T23:15:32.842 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:32.842 INFO:teuthology.orchestra.run.trial189.stdout:Total 401 kB/s | 170 kB 00:00 2026-02-22T23:15:32.842 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:32.845 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:32.845 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:32.876 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:32.877 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:32.897 INFO:teuthology.orchestra.run.trial006.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-22T23:15:32.899 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:32.912 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-22T23:15:32.912 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:32.913 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-22T23:15:32.913 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:32.932 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:32.949 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:33.027 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:33.087 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:33.087 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:33.087 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:33.087 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:33.087 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:33.088 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:33.116 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:15:33.148 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:40 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:33.213 INFO:teuthology.orchestra.run.trial040.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:33.232 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:33.232 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:15:33.232 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:33.253 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-fuse 2026-02-22T23:15:33.426 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:00:49 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:33.509 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:33.510 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:33.510 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:15:33.510 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:33.510 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:33.510 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout:Installing dependencies: 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout:Installing weak dependencies: 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout:Install 3 Packages 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 107 M 2026-02-22T23:15:33.511 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 654 M 2026-02-22T23:15:33.512 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:33.562 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:40 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:33.623 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:33.645 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:33.646 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:33.646 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:33.646 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:33.646 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:33.646 INFO:teuthology.orchestra.run.trial040.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 85 k 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 227 k 2026-02-22T23:15:33.647 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:33.650 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-22T23:15:33.650 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:33.650 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-22T23:15:33.650 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:33.650 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:15:33.650 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:33.708 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:33.710 INFO:teuthology.orchestra.run.trial006.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-22T23:15:33.716 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-22T23:15:33.736 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-22T23:15:33.737 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:34.004 INFO:teuthology.orchestra.run.trial040.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 238 kB/s | 85 kB 00:00 2026-02-22T23:15:34.005 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:34.005 INFO:teuthology.orchestra.run.trial040.stdout:Total 237 kB/s | 85 kB 00:00 2026-02-22T23:15:34.005 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:34.008 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:34.008 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:34.039 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:34.039 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:34.092 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:34.109 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:34.191 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:34.234 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-22T23:15:34.237 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:34.250 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:34.250 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:34.250 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:34.250 INFO:teuthology.orchestra.run.trial040.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:34.251 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:34.251 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:34.281 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-mirror 2026-02-22T23:15:34.592 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:41 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:34.677 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:34.678 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:34.678 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:34.678 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:34.678 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 3.1 M 2026-02-22T23:15:34.678 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 13 M 2026-02-22T23:15:34.678 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:34.694 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-22T23:15:34.694 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:34.707 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-22T23:15:34.707 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:34.762 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-22T23:15:34.812 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-22T23:15:34.813 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:34.835 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-22T23:15:34.835 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:34.835 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-22T23:15:34.835 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:34.835 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:15:34.835 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:34.848 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:35.066 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-22T23:15:35.068 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:35.090 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-22T23:15:35.090 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:35.090 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-22T23:15:35.091 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:35.091 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:15:35.091 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:35.194 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:35.215 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-22T23:15:35.216 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:35.216 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-22T23:15:35.216 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:35.216 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:15:35.216 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:35.234 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-22T23:15:35.886 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-22T23:15:35.887 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-22T23:15:35.888 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-22T23:15:35.889 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 49/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 50/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-22T23:15:35.890 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-22T23:15:35.891 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-22T23:15:35.892 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-22T23:15:35.893 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-22T23:15:35.894 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-22T23:15:35.894 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-22T23:15:35.894 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-22T23:15:35.894 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-22T23:15:35.948 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-22T23:15:35.948 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:35.948 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:35.948 INFO:teuthology.orchestra.run.trial006.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-22T23:15:35.948 INFO:teuthology.orchestra.run.trial006.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:35.949 INFO:teuthology.orchestra.run.trial006.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-22T23:15:35.950 INFO:teuthology.orchestra.run.trial006.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: lua-5.4.4-4.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-22T23:15:35.951 INFO:teuthology.orchestra.run.trial006.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-22T23:15:35.952 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-22T23:15:35.953 INFO:teuthology.orchestra.run.trial006.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-ply-3.11-14.el9.noarch 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-22T23:15:35.954 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-22T23:15:35.955 INFO:teuthology.orchestra.run.trial006.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: unzip-6.0-59.el9.x86_64 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: zip-3.0-35.el9.x86_64 2026-02-22T23:15:35.956 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:35.957 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:36.003 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-base 2026-02-22T23:15:36.081 INFO:teuthology.orchestra.run.trial040.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 2.2 MB/s | 3.1 MB 00:01 2026-02-22T23:15:36.082 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:36.082 INFO:teuthology.orchestra.run.trial040.stdout:Total 2.2 MB/s | 3.1 MB 00:01 2026-02-22T23:15:36.082 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:36.086 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:36.087 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:36.127 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:36.127 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:36.315 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:32 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:36.321 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:36.323 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:36.347 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:36.347 INFO:teuthology.orchestra.run.trial040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:36.347 INFO:teuthology.orchestra.run.trial040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-22T23:15:36.347 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:36.348 INFO:teuthology.orchestra.run.trial040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:36.348 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:36.380 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:36.398 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:36.399 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:36.399 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:36.415 INFO:teuthology.orchestra.run.trial189.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 3.7 MB/s | 11 MB 00:02 2026-02-22T23:15:36.420 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install cephadm 2026-02-22T23:15:36.643 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:36.644 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:36.644 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:36.644 INFO:teuthology.orchestra.run.trial040.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:36.644 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:36.644 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:36.671 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-nbd 2026-02-22T23:15:36.726 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:32 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:36.793 INFO:teuthology.orchestra.run.trial006.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:36.811 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:36.812 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:36.812 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:36.834 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-immutable-object-cache 2026-02-22T23:15:36.986 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:36.999 INFO:teuthology.orchestra.run.trial189.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 7.0 MB/s | 24 MB 00:03 2026-02-22T23:15:37.069 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:37.070 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:37.071 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:37.071 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:37.071 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 170 k 2026-02-22T23:15:37.071 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 490 k 2026-02-22T23:15:37.071 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:37.143 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:33 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:37.227 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:37.227 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:37.227 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:15:37.227 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:37.227 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:37.227 INFO:teuthology.orchestra.run.trial006.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 145 k 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 431 k 2026-02-22T23:15:37.228 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:37.268 INFO:teuthology.orchestra.run.trial040.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 860 kB/s | 170 kB 00:00 2026-02-22T23:15:37.269 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:37.269 INFO:teuthology.orchestra.run.trial040.stdout:Total 856 kB/s | 170 kB 00:00 2026-02-22T23:15:37.269 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:37.273 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:37.273 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:37.304 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:37.304 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:37.360 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:37.376 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:37.429 INFO:teuthology.orchestra.run.trial006.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 719 kB/s | 145 kB 00:00 2026-02-22T23:15:37.431 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:37.431 INFO:teuthology.orchestra.run.trial006.stdout:Total 709 kB/s | 145 kB 00:00 2026-02-22T23:15:37.431 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:37.435 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:37.435 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:37.457 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:37.467 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:37.467 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:37.515 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:37.516 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:37.516 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:37.516 INFO:teuthology.orchestra.run.trial040.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:37.516 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:37.516 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:37.522 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:37.523 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:37.543 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:15:37.546 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:37.547 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:37.547 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-22T23:15:37.547 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:37.862 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-22T23:15:37.862 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:37.863 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:37.863 INFO:teuthology.orchestra.run.trial006.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:37.863 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:37.863 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:37.864 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:44 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:37.891 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr 2026-02-22T23:15:37.946 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:37.947 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:37.947 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:15:37.947 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:37.947 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:37.947 INFO:teuthology.orchestra.run.trial040.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout:Installing dependencies: 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout:Installing weak dependencies: 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout:Install 3 Packages 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 107 M 2026-02-22T23:15:37.948 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 654 M 2026-02-22T23:15:37.949 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:38.205 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:34 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:38.270 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:38.288 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:38.289 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:38.289 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:38.310 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-dashboard 2026-02-22T23:15:38.625 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:34 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:38.691 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:38.709 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:38.709 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:38.710 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:38.731 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-22T23:15:39.039 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:35 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:39.105 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:39.123 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:39.124 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:39.124 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:39.145 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-rook 2026-02-22T23:15:39.455 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:35 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:39.521 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:39.541 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:39.542 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:39.542 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:39.564 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-cephadm 2026-02-22T23:15:39.872 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:35 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:39.940 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:39.958 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:39.959 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:39.959 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:39.980 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-fuse 2026-02-22T23:15:40.095 INFO:teuthology.orchestra.run.trial040.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 5.0 MB/s | 11 MB 00:02 2026-02-22T23:15:40.289 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:36 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:40.372 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:40.373 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:40.374 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:15:40.374 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:40.374 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 1.2 M 2026-02-22T23:15:40.374 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 3.6 M 2026-02-22T23:15:40.374 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:41.215 INFO:teuthology.orchestra.run.trial040.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 7.5 MB/s | 24 MB 00:03 2026-02-22T23:15:41.400 INFO:teuthology.orchestra.run.trial006.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.1 MB/s | 1.2 MB 00:01 2026-02-22T23:15:41.402 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:41.402 INFO:teuthology.orchestra.run.trial006.stdout:Total 1.1 MB/s | 1.2 MB 00:01 2026-02-22T23:15:41.402 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:41.406 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:41.406 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:41.443 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:41.443 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:41.545 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:41.563 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:41.809 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:41.868 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:41.868 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:41.868 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:41.868 INFO:teuthology.orchestra.run.trial006.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:41.868 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:41.869 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:41.896 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-volume 2026-02-22T23:15:42.204 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:42.271 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-22T23:15:42.290 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:42.290 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:42.290 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:42.311 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados-devel 2026-02-22T23:15:42.622 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:38 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:42.707 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:42.707 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:42.707 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:15:42.707 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:42.707 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:42.707 INFO:teuthology.orchestra.run.trial006.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 126 k 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 456 k 2026-02-22T23:15:42.708 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:43.061 INFO:teuthology.orchestra.run.trial006.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 357 kB/s | 126 kB 00:00 2026-02-22T23:15:43.061 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:43.062 INFO:teuthology.orchestra.run.trial006.stdout:Total 356 kB/s | 126 kB 00:00 2026-02-22T23:15:43.062 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:43.064 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:43.064 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:43.097 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:43.097 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:43.152 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:43.169 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:43.251 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:43.309 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:43.309 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:43.309 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:43.309 INFO:teuthology.orchestra.run.trial006.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:43.309 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:43.309 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:43.337 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install libcephfs2 2026-02-22T23:15:43.379 INFO:teuthology.orchestra.run.trial040.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 13 MB/s | 72 MB 00:05 2026-02-22T23:15:43.381 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:43.381 INFO:teuthology.orchestra.run.trial040.stdout:Total 20 MB/s | 107 MB 00:05 2026-02-22T23:15:43.381 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:15:43.386 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:15:43.386 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:15:43.652 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:39 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:43.718 INFO:teuthology.orchestra.run.trial006.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:43.737 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:43.737 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:43.737 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:43.758 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install libcephfs-devel 2026-02-22T23:15:43.764 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:15:43.764 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:15:44.067 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:40 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:44.136 INFO:teuthology.orchestra.run.trial080.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 33 MB/s | 474 MB 00:14 2026-02-22T23:15:44.136 INFO:teuthology.orchestra.run.trial189.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 6.8 MB/s | 72 MB 00:10 2026-02-22T23:15:44.137 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:44.137 INFO:teuthology.orchestra.run.trial080.stdout:Total 33 MB/s | 474 MB 00:14 2026-02-22T23:15:44.137 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:15:44.138 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:44.138 INFO:teuthology.orchestra.run.trial189.stdout:Total 10 MB/s | 107 MB 00:10 2026-02-22T23:15:44.139 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:15:44.139 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:15:44.139 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:15:44.144 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:15:44.144 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:15:44.152 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:44.152 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:44.152 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 34 k 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 153 k 2026-02-22T23:15:44.153 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:44.217 INFO:teuthology.orchestra.run.trial006.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 522 kB/s | 34 kB 00:00 2026-02-22T23:15:44.218 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:44.218 INFO:teuthology.orchestra.run.trial006.stdout:Total 515 kB/s | 34 kB 00:00 2026-02-22T23:15:44.218 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:44.220 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:44.220 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:44.222 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:44.222 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:44.243 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:44.258 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:44.314 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:44.372 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:44.372 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:44.372 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:44.372 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:44.373 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:44.373 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:44.402 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados2 2026-02-22T23:15:44.525 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:15:44.526 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:15:44.719 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:40 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:44.785 INFO:teuthology.orchestra.run.trial006.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:44.803 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:44.804 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:44.804 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:44.826 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librbd1 2026-02-22T23:15:45.137 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:41 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:45.205 INFO:teuthology.orchestra.run.trial006.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:45.223 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:45.224 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:45.224 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:45.245 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rados 2026-02-22T23:15:45.549 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:15:45.549 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:15:45.556 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:41 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:45.624 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:45.642 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:45.643 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:45.643 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:45.664 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rgw 2026-02-22T23:15:45.988 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:41 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:46.054 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:46.072 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:46.073 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:46.073 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:46.095 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-cephfs 2026-02-22T23:15:46.219 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:15:46.407 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:46.473 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:46.491 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:46.492 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:46.492 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:46.513 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rbd 2026-02-22T23:15:46.823 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:42 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:46.891 INFO:teuthology.orchestra.run.trial006.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:15:46.909 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:46.910 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:15:46.910 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:46.932 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-fuse 2026-02-22T23:15:46.981 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:15:47.248 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:43 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:47.332 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:47.332 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:47.332 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:47.332 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:47.332 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:47.332 INFO:teuthology.orchestra.run.trial006.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-22T23:15:47.332 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:47.333 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:47.333 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:47.333 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:15:47.333 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:47.333 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 85 k 2026-02-22T23:15:47.333 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 227 k 2026-02-22T23:15:47.333 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:47.487 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-22T23:15:47.597 INFO:teuthology.orchestra.run.trial006.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 322 kB/s | 85 kB 00:00 2026-02-22T23:15:47.599 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:47.599 INFO:teuthology.orchestra.run.trial006.stdout:Total 319 kB/s | 85 kB 00:00 2026-02-22T23:15:47.599 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:47.603 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:47.603 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:47.634 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:47.634 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:47.685 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:47.702 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:47.778 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:47.836 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:47.837 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:47.837 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:47.837 INFO:teuthology.orchestra.run.trial006.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:47.837 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:47.837 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:47.866 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-mirror 2026-02-22T23:15:48.192 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:44 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:48.275 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:48.276 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:48.276 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:15:48.276 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:48.276 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:48.276 INFO:teuthology.orchestra.run.trial006.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-22T23:15:48.276 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:48.276 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:48.277 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:48.277 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:15:48.277 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:48.277 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 3.1 M 2026-02-22T23:15:48.277 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 13 M 2026-02-22T23:15:48.277 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:48.307 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-22T23:15:48.429 INFO:teuthology.orchestra.run.trial006.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 20 MB/s | 3.1 MB 00:00 2026-02-22T23:15:48.430 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:48.430 INFO:teuthology.orchestra.run.trial006.stdout:Total 20 MB/s | 3.1 MB 00:00 2026-02-22T23:15:48.430 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:48.434 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:48.434 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:48.475 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:48.475 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:48.666 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:48.667 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:48.691 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:48.691 INFO:teuthology.orchestra.run.trial006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:15:48.691 INFO:teuthology.orchestra.run.trial006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-22T23:15:48.691 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:48.691 INFO:teuthology.orchestra.run.trial006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:15:48.692 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:49.020 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:49.020 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:49.020 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:49.020 INFO:teuthology.orchestra.run.trial006.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:49.020 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:49.021 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:49.049 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-nbd 2026-02-22T23:15:49.364 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:45 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:49.449 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:49.449 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:49.449 INFO:teuthology.orchestra.run.trial006.stdout: Package Architecture Version Repository Size 2026-02-22T23:15:49.449 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:49.449 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:49.449 INFO:teuthology.orchestra.run.trial006.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 170 k 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 490 k 2026-02-22T23:15:49.450 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:49.536 INFO:teuthology.orchestra.run.trial006.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.9 MB/s | 170 kB 00:00 2026-02-22T23:15:49.537 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:49.537 INFO:teuthology.orchestra.run.trial006.stdout:Total 1.9 MB/s | 170 kB 00:00 2026-02-22T23:15:49.537 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:49.541 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:49.541 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:49.572 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:49.572 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:49.629 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:49.645 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:49.734 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:49.793 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:15:49.793 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:49.793 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:15:49.793 INFO:teuthology.orchestra.run.trial006.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:49.793 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:49.793 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:15:49.820 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:15:50.131 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:46 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:15:50.217 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:15:50.218 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:50.218 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:15:50.218 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:50.218 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout:Installing dependencies: 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout:Installing weak dependencies: 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout:Install 3 Packages 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:15:50.219 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 107 M 2026-02-22T23:15:50.220 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 654 M 2026-02-22T23:15:50.220 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:15:51.001 INFO:teuthology.orchestra.run.trial006.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 14 MB/s | 11 MB 00:00 2026-02-22T23:15:51.183 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:15:51.202 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:15:51.443 INFO:teuthology.orchestra.run.trial006.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 20 MB/s | 24 MB 00:01 2026-02-22T23:15:51.737 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:15:51.737 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-22T23:15:51.737 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:15:51.795 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-22T23:15:51.796 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:51.796 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:15:51.796 INFO:teuthology.orchestra.run.trial040.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:51.796 INFO:teuthology.orchestra.run.trial040.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:51.796 INFO:teuthology.orchestra.run.trial040.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:51.796 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:51.796 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:15:51.839 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-common-debuginfo 2026-02-22T23:15:52.156 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:00:59 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:15:52.185 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:15:52.204 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:15:52.238 INFO:teuthology.orchestra.run.trial006.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 36 MB/s | 72 MB 00:02 2026-02-22T23:15:52.239 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial006.stdout:Total 53 MB/s | 107 MB 00:02 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial040.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-22T23:15:52.240 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:52.241 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:15:52.241 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:15:52.241 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:15:52.241 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:15:52.241 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 474 M 2026-02-22T23:15:52.241 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 2.0 G 2026-02-22T23:15:52.241 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:15:52.245 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:15:52.245 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:15:52.624 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:15:52.624 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:15:52.764 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:15:52.765 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-22T23:15:52.765 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:15:52.823 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-22T23:15:52.824 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:52.824 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:15:52.824 INFO:teuthology.orchestra.run.trial189.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:52.824 INFO:teuthology.orchestra.run.trial189.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:52.824 INFO:teuthology.orchestra.run.trial189.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:15:52.824 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:52.824 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:15:52.865 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-common-debuginfo 2026-02-22T23:15:53.183 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:09 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:15:53.266 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:15:53.266 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:53.266 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 474 M 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 2.0 G 2026-02-22T23:15:53.267 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:15:55.096 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:15:56.368 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-22T23:16:00.061 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:16:00.081 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:16:00.647 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-22T23:16:00.648 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-22T23:16:00.648 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-22T23:16:00.708 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-22T23:16:00.708 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:00.708 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:16:00.708 INFO:teuthology.orchestra.run.trial006.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:00.708 INFO:teuthology.orchestra.run.trial006.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:00.708 INFO:teuthology.orchestra.run.trial006.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:00.708 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:00.709 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:16:00.748 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-common-debuginfo 2026-02-22T23:16:01.064 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:00:57 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:16:01.147 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:16:01.147 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 474 M 2026-02-22T23:16:01.148 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 2.0 G 2026-02-22T23:16:01.149 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:16:04.164 INFO:teuthology.orchestra.run.trial040.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 40 MB/s | 474 MB 00:11 2026-02-22T23:16:04.165 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:04.165 INFO:teuthology.orchestra.run.trial040.stdout:Total 40 MB/s | 474 MB 00:11 2026-02-22T23:16:04.165 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:16:04.167 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:16:04.167 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:16:05.576 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:16:05.576 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:16:09.064 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:16:09.082 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:10.621 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:10.679 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:10.679 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:10.679 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:16:10.679 INFO:teuthology.orchestra.run.trial080.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:10.679 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:10.679 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:16:10.749 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-22T23:16:11.063 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:01:39 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:16:11.146 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:16:11.146 INFO:teuthology.orchestra.run.trial080.stdout:======================================================================================= 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout:======================================================================================= 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout:======================================================================================= 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:11.147 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 1.5 M 2026-02-22T23:16:11.148 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 7.6 M 2026-02-22T23:16:11.148 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:16:11.582 INFO:teuthology.orchestra.run.trial080.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 3.5 MB/s | 1.5 MB 00:00 2026-02-22T23:16:11.583 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:11.583 INFO:teuthology.orchestra.run.trial080.stdout:Total 3.5 MB/s | 1.5 MB 00:00 2026-02-22T23:16:11.583 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:16:11.584 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:16:11.584 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:16:11.606 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:16:11.606 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:16:11.719 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:16:11.736 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:11.796 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:11.853 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:11.853 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:11.854 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:16:11.854 INFO:teuthology.orchestra.run.trial080.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:11.854 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:11.854 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:16:11.880 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-22T23:16:12.192 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:01:40 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:16:12.279 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:16:12.279 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:16:12.279 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:16:12.279 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:16:12.279 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:16:12.279 INFO:teuthology.orchestra.run.trial080.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-22T23:16:12.279 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:12.280 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:16:12.280 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:16:12.280 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:16:12.280 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:12.280 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 200 M 2026-02-22T23:16:12.280 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 893 M 2026-02-22T23:16:12.280 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:16:16.211 INFO:teuthology.orchestra.run.trial189.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 21 MB/s | 474 MB 00:22 2026-02-22T23:16:16.212 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:16.212 INFO:teuthology.orchestra.run.trial189.stdout:Total 21 MB/s | 474 MB 00:22 2026-02-22T23:16:16.212 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:16:16.213 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:16:16.214 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:16:17.621 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:16:17.621 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:16:21.471 INFO:teuthology.orchestra.run.trial080.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 22 MB/s | 200 MB 00:09 2026-02-22T23:16:21.472 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:21.472 INFO:teuthology.orchestra.run.trial080.stdout:Total 22 MB/s | 200 MB 00:09 2026-02-22T23:16:21.472 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:16:21.474 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:16:21.474 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:16:22.079 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:16:22.079 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:16:26.305 INFO:teuthology.orchestra.run.trial006.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 19 MB/s | 474 MB 00:25 2026-02-22T23:16:26.306 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:26.306 INFO:teuthology.orchestra.run.trial006.stdout:Total 19 MB/s | 474 MB 00:25 2026-02-22T23:16:26.306 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:16:26.308 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:16:26.308 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:16:27.717 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:16:27.717 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:16:29.165 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:16:29.183 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:30.706 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:30.763 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:30.764 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:30.764 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:16:30.764 INFO:teuthology.orchestra.run.trial040.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:30.764 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:30.764 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:16:30.833 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-22T23:16:31.149 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:01:38 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:16:31.235 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:16:31.236 INFO:teuthology.orchestra.run.trial040.stdout:======================================================================================= 2026-02-22T23:16:31.236 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:16:31.236 INFO:teuthology.orchestra.run.trial040.stdout:======================================================================================= 2026-02-22T23:16:31.236 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:16:31.237 INFO:teuthology.orchestra.run.trial040.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-22T23:16:31.237 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:31.237 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:16:31.237 INFO:teuthology.orchestra.run.trial040.stdout:======================================================================================= 2026-02-22T23:16:31.237 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:16:31.237 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:31.238 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 1.5 M 2026-02-22T23:16:31.238 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 7.6 M 2026-02-22T23:16:31.238 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:16:31.360 INFO:teuthology.orchestra.run.trial040.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 12 MB/s | 1.5 MB 00:00 2026-02-22T23:16:31.360 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:31.360 INFO:teuthology.orchestra.run.trial040.stdout:Total 12 MB/s | 1.5 MB 00:00 2026-02-22T23:16:31.360 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:16:31.362 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:16:31.362 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:16:31.384 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:16:31.384 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:16:31.496 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:16:31.514 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:31.573 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:31.631 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:31.631 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:31.631 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:16:31.631 INFO:teuthology.orchestra.run.trial040.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:31.631 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:31.631 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:16:31.657 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-22T23:16:31.971 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:01:38 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:16:32.056 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:16:32.057 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:16:32.057 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:16:32.057 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:16:32.057 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:16:32.057 INFO:teuthology.orchestra.run.trial040.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-22T23:16:32.057 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:32.057 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:16:32.058 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:16:32.058 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:16:32.058 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:32.058 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 200 M 2026-02-22T23:16:32.058 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 893 M 2026-02-22T23:16:32.058 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:16:32.105 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:16:32.123 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:16:32.826 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:16:32.884 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:16:32.884 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:32.884 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:16:32.885 INFO:teuthology.orchestra.run.trial080.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:32.885 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:32.885 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:16:32.929 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-test-debuginfo 2026-02-22T23:16:33.244 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:02:01 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:16:33.327 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:16:33.327 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:16:33.327 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:16:33.327 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:16:33.327 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:16:33.327 INFO:teuthology.orchestra.run.trial080.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 729 M 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 3.6 G 2026-02-22T23:16:33.328 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:16:36.736 INFO:teuthology.orchestra.run.trial040.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 43 MB/s | 200 MB 00:04 2026-02-22T23:16:36.737 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:36.737 INFO:teuthology.orchestra.run.trial040.stdout:Total 43 MB/s | 200 MB 00:04 2026-02-22T23:16:36.737 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:16:36.739 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:16:36.739 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:16:37.344 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:16:37.344 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:16:41.151 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:16:41.169 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:42.680 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:42.739 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:42.739 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:42.739 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:16:42.739 INFO:teuthology.orchestra.run.trial189.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:42.739 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:42.739 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:16:42.809 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-22T23:16:43.124 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:59 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:16:43.206 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:======================================================================================= 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:======================================================================================= 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:======================================================================================= 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 1.5 M 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 7.6 M 2026-02-22T23:16:43.207 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:16:43.337 INFO:teuthology.orchestra.run.trial189.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 12 MB/s | 1.5 MB 00:00 2026-02-22T23:16:43.337 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:43.338 INFO:teuthology.orchestra.run.trial189.stdout:Total 12 MB/s | 1.5 MB 00:00 2026-02-22T23:16:43.338 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:16:43.339 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:16:43.339 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:16:43.361 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:16:43.362 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:16:43.476 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:16:43.493 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:43.553 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:43.611 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:43.611 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:43.611 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:16:43.611 INFO:teuthology.orchestra.run.trial189.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:43.611 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:43.612 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:16:43.638 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-22T23:16:43.949 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:01:59 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:16:44.032 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:16:44.033 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:16:44.033 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:16:44.033 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:16:44.033 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:16:44.033 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 200 M 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 893 M 2026-02-22T23:16:44.034 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:16:47.388 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:16:47.406 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:16:48.108 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:16:48.167 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:16:48.167 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:48.167 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:16:48.167 INFO:teuthology.orchestra.run.trial040.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:48.167 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:48.167 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:16:48.214 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-test-debuginfo 2026-02-22T23:16:48.528 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:01:55 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:16:48.611 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:16:48.612 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:16:48.612 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:16:48.612 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 729 M 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 3.6 G 2026-02-22T23:16:48.613 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:16:49.542 INFO:teuthology.orchestra.run.trial189.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 36 MB/s | 200 MB 00:05 2026-02-22T23:16:49.543 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:49.543 INFO:teuthology.orchestra.run.trial189.stdout:Total 36 MB/s | 200 MB 00:05 2026-02-22T23:16:49.543 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:16:49.545 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:16:49.545 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:16:50.151 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:16:50.151 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:16:51.732 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:16:51.750 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:53.263 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:53.322 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:16:53.322 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:53.322 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:16:53.322 INFO:teuthology.orchestra.run.trial006.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:53.322 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:53.323 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:16:53.395 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-22T23:16:53.713 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:01:49 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:16:53.797 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:16:53.798 INFO:teuthology.orchestra.run.trial006.stdout:======================================================================================= 2026-02-22T23:16:53.798 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout:======================================================================================= 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout:======================================================================================= 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 1.5 M 2026-02-22T23:16:53.799 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 7.6 M 2026-02-22T23:16:53.800 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:16:54.288 INFO:teuthology.orchestra.run.trial006.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 3.1 MB/s | 1.5 MB 00:00 2026-02-22T23:16:54.289 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:16:54.289 INFO:teuthology.orchestra.run.trial006.stdout:Total 3.1 MB/s | 1.5 MB 00:00 2026-02-22T23:16:54.289 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:16:54.291 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:16:54.291 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:16:54.313 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:16:54.314 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:16:54.428 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:16:54.445 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:54.505 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:54.564 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-22T23:16:54.564 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:54.564 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:16:54.564 INFO:teuthology.orchestra.run.trial006.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:16:54.564 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:54.565 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:16:54.593 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-22T23:16:54.907 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:01:50 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:16:54.992 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:16:54.992 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:16:54.992 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 200 M 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 893 M 2026-02-22T23:16:54.993 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:17:00.628 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:17:00.646 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:17:01.306 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:17:01.366 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:17:01.366 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:17:01.366 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:17:01.366 INFO:teuthology.orchestra.run.trial189.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:17:01.367 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:17:01.367 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:17:01.413 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-test-debuginfo 2026-02-22T23:17:01.722 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:02:17 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:17:01.805 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:17:01.805 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:17:01.805 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 729 M 2026-02-22T23:17:01.806 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 3.6 G 2026-02-22T23:17:01.807 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:17:06.894 INFO:teuthology.orchestra.run.trial006.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 17 MB/s | 200 MB 00:11 2026-02-22T23:17:06.895 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:17:06.895 INFO:teuthology.orchestra.run.trial006.stdout:Total 17 MB/s | 200 MB 00:11 2026-02-22T23:17:06.896 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:17:06.897 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:17:06.897 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:17:07.504 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:17:07.505 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:17:17.811 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:17:17.829 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:17:18.473 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:17:18.532 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-22T23:17:18.532 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:17:18.532 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:17:18.532 INFO:teuthology.orchestra.run.trial006.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:17:18.533 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:17:18.533 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:17:18.578 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-test-debuginfo 2026-02-22T23:17:18.891 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:02:14 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:17:18.974 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:17:18.975 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:17:18.975 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 729 M 2026-02-22T23:17:18.976 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 3.6 G 2026-02-22T23:17:18.977 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:17:30.150 INFO:teuthology.orchestra.run.trial080.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 13 MB/s | 729 MB 00:56 2026-02-22T23:17:30.151 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:17:30.151 INFO:teuthology.orchestra.run.trial080.stdout:Total 13 MB/s | 729 MB 00:56 2026-02-22T23:17:30.151 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:17:30.154 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:17:30.154 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:17:30.658 INFO:teuthology.orchestra.run.trial040.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 17 MB/s | 729 MB 00:42 2026-02-22T23:17:30.659 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:17:30.659 INFO:teuthology.orchestra.run.trial040.stdout:Total 17 MB/s | 729 MB 00:42 2026-02-22T23:17:30.659 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:17:30.662 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:17:30.662 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:17:32.312 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:17:32.313 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:17:32.819 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:17:32.819 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:17:45.490 INFO:teuthology.orchestra.run.trial006.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 28 MB/s | 729 MB 00:26 2026-02-22T23:17:45.491 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:17:45.491 INFO:teuthology.orchestra.run.trial006.stdout:Total 28 MB/s | 729 MB 00:26 2026-02-22T23:17:45.491 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:17:45.494 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:17:45.494 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:17:47.662 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:17:47.663 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:17:50.055 INFO:teuthology.orchestra.run.trial189.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 15 MB/s | 729 MB 00:48 2026-02-22T23:17:50.056 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:17:50.056 INFO:teuthology.orchestra.run.trial189.stdout:Total 15 MB/s | 729 MB 00:48 2026-02-22T23:17:50.056 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:17:50.058 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:17:50.059 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:17:52.218 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:17:52.219 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:18:09.997 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:18:10.018 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:10.544 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:10.569 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:10.696 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:10.754 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:10.754 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:10.754 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:18:10.754 INFO:teuthology.orchestra.run.trial080.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:10.754 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:10.754 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:18:10.850 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:18:11.165 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:03:39 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:18:11.228 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:18:11.247 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:18:11.248 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-22T23:18:11.248 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:18:11.253 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:11.270 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-debuginfo 2026-02-22T23:18:11.310 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:11.310 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:11.310 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:11.310 INFO:teuthology.orchestra.run.trial040.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:11.311 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:11.311 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:11.407 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:18:11.580 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:03:39 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:18:11.663 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 18 M 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 110 M 2026-02-22T23:18:11.664 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:18:11.729 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:03:18 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:11.793 INFO:teuthology.orchestra.run.trial040.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:18:11.811 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:11.812 INFO:teuthology.orchestra.run.trial040.stdout:Nothing to do. 2026-02-22T23:18:11.812 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:11.834 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mgr-debuginfo 2026-02-22T23:18:12.120 INFO:teuthology.orchestra.run.trial080.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 40 MB/s | 18 MB 00:00 2026-02-22T23:18:12.120 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:12.120 INFO:teuthology.orchestra.run.trial080.stdout:Total 39 MB/s | 18 MB 00:00 2026-02-22T23:18:12.120 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:18:12.122 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:18:12.122 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:18:12.150 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:03:19 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:12.193 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:18:12.193 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 18 M 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 110 M 2026-02-22T23:18:12.233 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:18:12.664 INFO:teuthology.orchestra.run.trial040.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 42 MB/s | 18 MB 00:00 2026-02-22T23:18:12.665 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:12.665 INFO:teuthology.orchestra.run.trial040.stdout:Total 42 MB/s | 18 MB 00:00 2026-02-22T23:18:12.665 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:18:12.666 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:18:12.666 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:18:12.738 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:18:12.738 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:18:13.224 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:18:13.242 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:13.368 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:13.425 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:13.425 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:13.426 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:18:13.426 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:13.426 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:13.426 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:18:13.453 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mds-debuginfo 2026-02-22T23:18:13.754 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:13.764 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:03:41 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:18:13.772 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:13.848 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:18:13.849 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:13.850 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 55 M 2026-02-22T23:18:13.850 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 211 M 2026-02-22T23:18:13.850 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:18:13.908 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:13.966 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:13.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:13.966 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:13.966 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:13.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:13.966 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:13.995 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mds-debuginfo 2026-02-22T23:18:14.305 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:03:21 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:14.389 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:14.389 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 55 M 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 211 M 2026-02-22T23:18:14.390 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:18:16.422 INFO:teuthology.orchestra.run.trial040.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 27 MB/s | 55 MB 00:02 2026-02-22T23:18:16.424 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:16.424 INFO:teuthology.orchestra.run.trial040.stdout:Total 27 MB/s | 55 MB 00:02 2026-02-22T23:18:16.424 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:18:16.425 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:18:16.426 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:18:16.604 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:18:16.604 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:18:17.194 INFO:teuthology.orchestra.run.trial080.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 16 MB/s | 55 MB 00:03 2026-02-22T23:18:17.195 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:17.195 INFO:teuthology.orchestra.run.trial080.stdout:Total 16 MB/s | 55 MB 00:03 2026-02-22T23:18:17.195 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:18:17.197 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:18:17.197 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:18:17.377 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:18:17.377 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:18:19.287 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:19.305 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:19.521 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:19.579 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:19.579 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:19.579 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:19.579 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:19.580 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:19.580 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:19.614 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-mon-debuginfo 2026-02-22T23:18:19.926 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:03:26 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:20.009 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:20.009 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 102 M 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 497 M 2026-02-22T23:18:20.010 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:18:20.047 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:18:20.065 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:20.259 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:20.317 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:20.317 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:20.317 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:18:20.317 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:20.317 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:20.318 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:18:20.349 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mon-debuginfo 2026-02-22T23:18:20.661 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:03:48 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:18:20.745 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:18:20.745 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:20.745 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:18:20.745 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:20.745 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:18:20.745 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 102 M 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 497 M 2026-02-22T23:18:20.746 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:18:22.059 INFO:teuthology.orchestra.run.trial040.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 50 MB/s | 102 MB 00:02 2026-02-22T23:18:22.060 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:22.060 INFO:teuthology.orchestra.run.trial040.stdout:Total 50 MB/s | 102 MB 00:02 2026-02-22T23:18:22.060 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:18:22.062 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:18:22.062 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:18:22.379 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:18:22.379 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:18:24.034 INFO:teuthology.orchestra.run.trial080.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 31 MB/s | 102 MB 00:03 2026-02-22T23:18:24.034 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:24.035 INFO:teuthology.orchestra.run.trial080.stdout:Total 31 MB/s | 102 MB 00:03 2026-02-22T23:18:24.035 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:18:24.037 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:18:24.037 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:18:24.353 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:18:24.353 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:18:25.516 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:18:25.602 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:27.597 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:27.615 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:27.655 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:27.714 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:27.715 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:27.715 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:18:27.715 INFO:teuthology.orchestra.run.trial006.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:27.715 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:27.715 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:18:27.815 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:18:27.993 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:28.051 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:28.051 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:28.051 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:28.052 INFO:teuthology.orchestra.run.trial040.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:28.052 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:28.052 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:28.090 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-osd-debuginfo 2026-02-22T23:18:28.123 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:03:24 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:18:28.187 INFO:teuthology.orchestra.run.trial006.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:18:28.206 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:18:28.206 INFO:teuthology.orchestra.run.trial006.stdout:Nothing to do. 2026-02-22T23:18:28.207 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:18:28.228 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mgr-debuginfo 2026-02-22T23:18:28.408 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:03:35 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:28.493 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:28.493 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:28.493 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:28.493 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:28.493 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:28.493 INFO:teuthology.orchestra.run.trial040.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-22T23:18:28.493 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:28.494 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:28.494 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:28.494 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:28.494 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:28.494 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 298 M 2026-02-22T23:18:28.494 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 1.4 G 2026-02-22T23:18:28.494 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:18:28.544 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:03:24 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:18:28.627 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:28.628 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:18:28.629 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:28.629 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 18 M 2026-02-22T23:18:28.629 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 110 M 2026-02-22T23:18:28.629 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:18:29.765 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:18:29.783 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:30.208 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:30.266 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:30.266 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:30.266 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:18:30.266 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:30.266 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:30.266 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:18:30.303 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-osd-debuginfo 2026-02-22T23:18:30.616 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:03:58 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:18:30.698 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:18:30.699 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 298 M 2026-02-22T23:18:30.699 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 1.4 G 2026-02-22T23:18:30.699 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:18:31.405 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:18:31.408 INFO:teuthology.orchestra.run.trial006.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 6.5 MB/s | 18 MB 00:02 2026-02-22T23:18:31.408 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:31.409 INFO:teuthology.orchestra.run.trial006.stdout:Total 6.5 MB/s | 18 MB 00:02 2026-02-22T23:18:31.409 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:18:31.411 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:18:31.411 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:18:31.426 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:31.481 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:18:31.482 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:18:32.418 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:32.478 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:32.478 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:32.478 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:18:32.478 INFO:teuthology.orchestra.run.trial189.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:32.478 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:32.478 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:18:32.506 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:18:32.524 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:32.576 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-base-debuginfo 2026-02-22T23:18:32.653 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:32.712 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:32.712 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:32.712 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:18:32.712 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:32.712 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:32.712 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:18:32.741 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mds-debuginfo 2026-02-22T23:18:32.891 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:03:48 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:18:32.955 INFO:teuthology.orchestra.run.trial189.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-22T23:18:32.974 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:18:32.974 INFO:teuthology.orchestra.run.trial189.stdout:Nothing to do. 2026-02-22T23:18:32.975 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:18:32.995 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mgr-debuginfo 2026-02-22T23:18:33.054 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:03:29 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:33.139 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 55 M 2026-02-22T23:18:33.140 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 211 M 2026-02-22T23:18:33.140 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:18:33.306 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:03:49 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:18:33.389 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:18:33.389 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:33.389 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:18:33.389 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:33.389 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 18 M 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 110 M 2026-02-22T23:18:33.390 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:18:35.741 INFO:teuthology.orchestra.run.trial189.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 7.7 MB/s | 18 MB 00:02 2026-02-22T23:18:35.742 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:35.742 INFO:teuthology.orchestra.run.trial189.stdout:Total 7.7 MB/s | 18 MB 00:02 2026-02-22T23:18:35.742 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:18:35.743 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:18:35.743 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:18:35.794 INFO:teuthology.orchestra.run.trial040.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 41 MB/s | 298 MB 00:07 2026-02-22T23:18:35.795 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:35.796 INFO:teuthology.orchestra.run.trial040.stdout:Total 41 MB/s | 298 MB 00:07 2026-02-22T23:18:35.796 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:18:35.797 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:18:35.798 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:18:35.814 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:18:35.814 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:18:36.686 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:18:36.687 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:18:36.830 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:18:36.847 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:36.975 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:37.034 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:37.034 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:37.034 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:18:37.034 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:37.034 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:37.035 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:18:37.063 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mds-debuginfo 2026-02-22T23:18:37.373 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:03:53 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:18:37.456 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:18:37.456 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:37.456 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:18:37.456 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:37.457 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:18:37.457 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:18:37.457 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:37.457 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:18:37.457 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:37.458 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:18:37.458 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:37.458 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 55 M 2026-02-22T23:18:37.458 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 211 M 2026-02-22T23:18:37.458 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:18:37.968 INFO:teuthology.orchestra.run.trial006.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 11 MB/s | 55 MB 00:04 2026-02-22T23:18:37.969 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:37.969 INFO:teuthology.orchestra.run.trial006.stdout:Total 11 MB/s | 55 MB 00:04 2026-02-22T23:18:37.969 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:18:37.970 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:18:37.970 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:18:38.149 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:18:38.150 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:18:38.948 INFO:teuthology.orchestra.run.trial189.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 37 MB/s | 55 MB 00:01 2026-02-22T23:18:38.949 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:38.949 INFO:teuthology.orchestra.run.trial189.stdout:Total 37 MB/s | 55 MB 00:01 2026-02-22T23:18:38.949 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:18:38.950 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:18:38.951 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:18:39.131 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:18:39.131 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:18:40.822 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:18:40.839 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:41.046 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:41.105 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:41.105 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:41.105 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:18:41.105 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:41.105 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:41.105 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:18:41.137 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-mon-debuginfo 2026-02-22T23:18:41.452 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:03:37 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:18:41.535 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:18:41.535 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:41.535 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:18:41.535 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:41.535 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:18:41.536 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-22T23:18:41.536 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:41.536 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:18:41.536 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:41.536 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:18:41.536 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:41.537 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 102 M 2026-02-22T23:18:41.537 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 497 M 2026-02-22T23:18:41.537 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:18:41.806 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:18:41.824 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:42.030 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:42.090 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:42.090 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:42.090 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:18:42.091 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:42.091 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:42.091 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:18:42.124 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-mon-debuginfo 2026-02-22T23:18:42.435 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:03:58 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 102 M 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 497 M 2026-02-22T23:18:42.517 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:18:48.287 INFO:teuthology.orchestra.run.trial006.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 15 MB/s | 102 MB 00:06 2026-02-22T23:18:48.288 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:48.288 INFO:teuthology.orchestra.run.trial006.stdout:Total 15 MB/s | 102 MB 00:06 2026-02-22T23:18:48.288 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:18:48.290 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:18:48.290 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:18:48.606 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:18:48.606 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:18:50.611 INFO:teuthology.orchestra.run.trial189.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 13 MB/s | 102 MB 00:08 2026-02-22T23:18:50.611 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:50.612 INFO:teuthology.orchestra.run.trial189.stdout:Total 13 MB/s | 102 MB 00:08 2026-02-22T23:18:50.612 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:18:50.614 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:18:50.614 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:18:50.931 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:18:50.931 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:18:51.056 INFO:teuthology.orchestra.run.trial080.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 15 MB/s | 298 MB 00:20 2026-02-22T23:18:51.056 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:51.056 INFO:teuthology.orchestra.run.trial080.stdout:Total 15 MB/s | 298 MB 00:20 2026-02-22T23:18:51.057 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:18:51.059 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:18:51.059 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:18:51.786 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:51.804 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:51.949 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:18:51.949 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:18:52.875 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:52.933 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:52.934 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:52.934 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:52.934 INFO:teuthology.orchestra.run.trial040.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:52.934 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:52.934 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:52.989 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install ceph-fuse-debuginfo 2026-02-22T23:18:53.301 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:00 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:53.384 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 6.8 M 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 36 M 2026-02-22T23:18:53.385 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:18:53.614 INFO:teuthology.orchestra.run.trial040.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 30 MB/s | 6.8 MB 00:00 2026-02-22T23:18:53.616 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:53.616 INFO:teuthology.orchestra.run.trial040.stdout:Total 30 MB/s | 6.8 MB 00:00 2026-02-22T23:18:53.616 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:18:53.617 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:18:53.617 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:18:53.654 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:18:53.655 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:18:53.817 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:18:53.835 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:54.053 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:54.071 INFO:teuthology.orchestra.run.trial040.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:54.154 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:54.213 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:18:54.213 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:54.213 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:54.213 INFO:teuthology.orchestra.run.trial040.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:54.213 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:54.213 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:54.242 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librados-devel-debuginfo 2026-02-22T23:18:54.243 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:54.303 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:54.303 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:54.303 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:18:54.303 INFO:teuthology.orchestra.run.trial006.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:54.303 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:54.303 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:18:54.338 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-osd-debuginfo 2026-02-22T23:18:54.552 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:01 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 112 k 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 493 k 2026-02-22T23:18:54.637 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:18:54.652 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:03:50 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:18:54.723 INFO:teuthology.orchestra.run.trial040.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 1.3 MB/s | 112 kB 00:00 2026-02-22T23:18:54.723 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:54.723 INFO:teuthology.orchestra.run.trial040.stdout:Total 1.3 MB/s | 112 kB 00:00 2026-02-22T23:18:54.723 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:18:54.725 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:18:54.725 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:18:54.735 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:18:54.736 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:54.736 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:18:54.736 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:54.736 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:18:54.736 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 298 M 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 1.4 G 2026-02-22T23:18:54.737 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:18:54.743 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:18:54.743 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:18:54.782 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:54.799 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:18:54.855 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:18:54.913 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:18:54.913 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:54.913 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:54.914 INFO:teuthology.orchestra.run.trial040.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:54.914 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:54.914 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:54.939 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install libcephfs2-debuginfo 2026-02-22T23:18:55.255 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:02 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:55.339 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:55.340 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:55.340 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:55.340 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:55.340 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 6.0 M 2026-02-22T23:18:55.340 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 31 M 2026-02-22T23:18:55.340 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:18:56.250 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:18:56.268 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:56.667 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:56.727 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:18:56.727 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:56.728 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:18:56.728 INFO:teuthology.orchestra.run.trial189.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:56.728 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:56.728 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:18:56.764 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-osd-debuginfo 2026-02-22T23:18:57.080 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:13 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:18:57.162 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:18:57.162 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:57.162 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:18:57.162 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:57.162 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 298 M 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 1.4 G 2026-02-22T23:18:57.163 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:18:57.345 INFO:teuthology.orchestra.run.trial040.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 3.0 MB/s | 6.0 MB 00:02 2026-02-22T23:18:57.345 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:18:57.345 INFO:teuthology.orchestra.run.trial040.stdout:Total 3.0 MB/s | 6.0 MB 00:02 2026-02-22T23:18:57.345 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:18:57.347 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:18:57.347 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:18:57.382 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:18:57.382 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:18:57.728 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:18:57.746 INFO:teuthology.orchestra.run.trial040.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:18:57.821 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:18:57.881 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:18:57.881 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:57.881 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:18:57.882 INFO:teuthology.orchestra.run.trial040.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:18:57.882 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:57.882 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:18:57.910 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librados2-debuginfo 2026-02-22T23:18:58.225 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:05 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:58.308 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:18:58.309 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:18:58.309 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:18:58.309 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:18:58.309 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 47 M 2026-02-22T23:18:58.309 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 302 M 2026-02-22T23:18:58.309 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:01.962 INFO:teuthology.orchestra.run.trial040.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 13 MB/s | 47 MB 00:03 2026-02-22T23:19:01.963 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:01.963 INFO:teuthology.orchestra.run.trial040.stdout:Total 13 MB/s | 47 MB 00:03 2026-02-22T23:19:01.963 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:01.965 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:01.965 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:02.122 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:02.122 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:02.426 INFO:teuthology.orchestra.run.trial006.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 39 MB/s | 298 MB 00:07 2026-02-22T23:19:02.427 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:02.427 INFO:teuthology.orchestra.run.trial006.stdout:Total 39 MB/s | 298 MB 00:07 2026-02-22T23:19:02.427 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:02.429 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:02.429 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:03.321 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:03.321 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:04.767 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:04.786 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:05.047 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:05.105 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:05.105 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:05.105 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:05.106 INFO:teuthology.orchestra.run.trial040.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:05.106 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:05.106 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:05.137 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install librbd1-debuginfo 2026-02-22T23:19:05.455 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:12 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:05.539 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:05.540 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:05.541 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:05.541 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 55 M 2026-02-22T23:19:05.541 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 374 M 2026-02-22T23:19:05.541 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:06.653 INFO:teuthology.orchestra.run.trial189.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 31 MB/s | 298 MB 00:09 2026-02-22T23:19:06.654 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:06.654 INFO:teuthology.orchestra.run.trial189.stdout:Total 31 MB/s | 298 MB 00:09 2026-02-22T23:19:06.654 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:06.656 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:06.656 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:06.971 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:06.988 INFO:teuthology.orchestra.run.trial040.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 38 MB/s | 55 MB 00:01 2026-02-22T23:19:06.988 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:06.989 INFO:teuthology.orchestra.run.trial040.stdout:Total 38 MB/s | 55 MB 00:01 2026-02-22T23:19:06.989 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:06.990 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:06.991 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:06.991 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:07.171 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:07.171 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:07.550 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:07.550 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:08.036 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:08.096 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:08.096 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:08.096 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:08.096 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:08.096 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:08.096 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:08.148 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-fuse-debuginfo 2026-02-22T23:19:08.462 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:36 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:08.546 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 6.8 M 2026-02-22T23:19:08.547 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 36 M 2026-02-22T23:19:08.547 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:08.766 INFO:teuthology.orchestra.run.trial080.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 31 MB/s | 6.8 MB 00:00 2026-02-22T23:19:08.767 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:08.767 INFO:teuthology.orchestra.run.trial080.stdout:Total 31 MB/s | 6.8 MB 00:00 2026-02-22T23:19:08.767 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:08.769 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:08.769 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:08.807 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:08.807 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:09.199 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:09.217 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:09.304 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:09.362 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:09.362 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:09.362 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:09.362 INFO:teuthology.orchestra.run.trial080.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:09.362 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:09.362 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:09.390 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librados-devel-debuginfo 2026-02-22T23:19:09.705 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:37 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:09.790 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:09.791 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:09.791 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:09.791 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 112 k 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 493 k 2026-02-22T23:19:09.792 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:09.877 INFO:teuthology.orchestra.run.trial080.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 1.3 MB/s | 112 kB 00:00 2026-02-22T23:19:09.878 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:09.878 INFO:teuthology.orchestra.run.trial080.stdout:Total 1.3 MB/s | 112 kB 00:00 2026-02-22T23:19:09.878 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:09.880 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:09.880 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:09.898 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:09.899 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:09.937 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:09.954 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:10.010 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:10.068 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:10.068 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:10.069 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:10.069 INFO:teuthology.orchestra.run.trial080.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:10.069 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:10.069 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:10.095 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install libcephfs2-debuginfo 2026-02-22T23:19:10.304 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:10.322 INFO:teuthology.orchestra.run.trial040.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:10.405 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:38 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:10.488 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 6.0 M 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 31 M 2026-02-22T23:19:10.489 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:10.666 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:10.693 INFO:teuthology.orchestra.run.trial080.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 29 MB/s | 6.0 MB 00:00 2026-02-22T23:19:10.694 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:10.695 INFO:teuthology.orchestra.run.trial080.stdout:Total 29 MB/s | 6.0 MB 00:00 2026-02-22T23:19:10.695 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:10.696 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:10.696 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:10.724 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:10.724 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:10.724 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:10.725 INFO:teuthology.orchestra.run.trial040.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:10.725 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:10.725 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:10.731 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:10.731 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:10.757 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-cephfs-debuginfo 2026-02-22T23:19:11.068 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:18 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:11.081 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:11.099 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:11.153 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:11.154 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 628 k 2026-02-22T23:19:11.154 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 2.1 M 2026-02-22T23:19:11.154 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:11.177 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:11.235 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:11.236 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:11.236 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:11.236 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:11.236 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:11.236 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:11.262 INFO:teuthology.orchestra.run.trial040.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 5.7 MB/s | 628 kB 00:00 2026-02-22T23:19:11.263 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:11.263 INFO:teuthology.orchestra.run.trial040.stdout:Total 5.7 MB/s | 628 kB 00:00 2026-02-22T23:19:11.263 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:11.264 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librados2-debuginfo 2026-02-22T23:19:11.265 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:11.265 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:11.283 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:11.284 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:11.348 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:11.365 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:11.424 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:11.483 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:11.483 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:11.483 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:11.484 INFO:teuthology.orchestra.run.trial040.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:11.484 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:11.484 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:11.510 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rados-debuginfo 2026-02-22T23:19:11.569 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:39 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:11.653 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:11.653 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:11.653 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:11.653 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:11.653 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:11.654 INFO:teuthology.orchestra.run.trial080.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-22T23:19:11.654 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:11.654 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:11.654 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:11.654 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:11.654 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:11.654 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 47 M 2026-02-22T23:19:11.655 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 302 M 2026-02-22T23:19:11.655 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:11.824 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:18 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:11.907 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:11.908 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 779 k 2026-02-22T23:19:11.908 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 2.8 M 2026-02-22T23:19:11.908 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:12.602 INFO:teuthology.orchestra.run.trial040.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 1.1 MB/s | 779 kB 00:00 2026-02-22T23:19:12.604 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:12.604 INFO:teuthology.orchestra.run.trial040.stdout:Total 1.1 MB/s | 779 kB 00:00 2026-02-22T23:19:12.604 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:12.606 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:12.606 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:12.626 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:12.626 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:12.632 INFO:teuthology.orchestra.run.trial080.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 48 MB/s | 47 MB 00:00 2026-02-22T23:19:12.632 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:12.632 INFO:teuthology.orchestra.run.trial080.stdout:Total 48 MB/s | 47 MB 00:00 2026-02-22T23:19:12.632 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:12.634 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:12.634 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:12.700 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:12.717 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:12.775 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:12.792 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:12.792 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:12.832 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:12.833 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:12.833 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:12.833 INFO:teuthology.orchestra.run.trial040.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:12.833 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:12.833 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:12.859 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rbd-debuginfo 2026-02-22T23:19:13.175 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:20 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:13.256 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:13.256 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:13.256 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:13.256 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:13.256 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 1.1 M 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 4.3 M 2026-02-22T23:19:13.257 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:14.212 INFO:teuthology.orchestra.run.trial040.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 1.2 MB/s | 1.1 MB 00:00 2026-02-22T23:19:14.213 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:14.213 INFO:teuthology.orchestra.run.trial040.stdout:Total 1.2 MB/s | 1.1 MB 00:00 2026-02-22T23:19:14.213 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:14.214 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:14.215 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:14.235 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:14.235 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:14.325 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:14.342 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:14.399 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:14.457 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:14.458 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:14.458 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:14.458 INFO:teuthology.orchestra.run.trial040.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:14.458 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:14.458 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:14.484 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install python3-rgw-debuginfo 2026-02-22T23:19:14.798 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:21 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:14.881 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:14.882 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:14.882 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 339 k 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 1.0 M 2026-02-22T23:19:14.883 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:15.391 INFO:teuthology.orchestra.run.trial040.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 667 kB/s | 339 kB 00:00 2026-02-22T23:19:15.392 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:15.392 INFO:teuthology.orchestra.run.trial040.stdout:Total 665 kB/s | 339 kB 00:00 2026-02-22T23:19:15.392 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:15.394 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:15.394 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:15.414 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:15.414 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:15.428 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:15.446 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:15.467 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:15.484 INFO:teuthology.orchestra.run.trial040.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:15.542 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:15.601 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:15.601 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:15.601 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:15.601 INFO:teuthology.orchestra.run.trial040.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:15.601 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:15.601 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:15.629 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-fuse-debuginfo 2026-02-22T23:19:15.724 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:15.782 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:15.782 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:15.783 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:15.783 INFO:teuthology.orchestra.run.trial080.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:15.783 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:15.783 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:15.816 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librbd1-debuginfo 2026-02-22T23:19:15.945 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:22 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:16.027 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:16.028 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:16.028 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:16.028 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:16.028 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 547 k 2026-02-22T23:19:16.028 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 3.0 M 2026-02-22T23:19:16.028 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:16.130 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:44 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:16.213 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:16.214 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:16.214 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:16.214 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:16.214 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:16.214 INFO:teuthology.orchestra.run.trial080.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:19:16.214 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:16.215 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:16.215 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:16.215 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:16.215 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:16.215 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 55 M 2026-02-22T23:19:16.215 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 374 M 2026-02-22T23:19:16.216 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:16.463 INFO:teuthology.orchestra.run.trial040.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 1.2 MB/s | 547 kB 00:00 2026-02-22T23:19:16.463 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:16.463 INFO:teuthology.orchestra.run.trial040.stdout:Total 1.2 MB/s | 547 kB 00:00 2026-02-22T23:19:16.463 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:16.466 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:16.466 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:16.485 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:16.485 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:16.548 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:16.565 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:16.621 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:16.681 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:16.681 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:16.681 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:16.681 INFO:teuthology.orchestra.run.trial040.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:16.681 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:16.681 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:16.709 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-mirror-debuginfo 2026-02-22T23:19:17.025 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:24 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:17.106 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 54 M 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 353 M 2026-02-22T23:19:17.107 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:18.920 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:18.938 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:19.360 INFO:teuthology.orchestra.run.trial080.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 17 MB/s | 55 MB 00:03 2026-02-22T23:19:19.361 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:19.361 INFO:teuthology.orchestra.run.trial080.stdout:Total 17 MB/s | 55 MB 00:03 2026-02-22T23:19:19.361 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:19.363 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:19.363 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:19.544 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:19.544 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:19.863 INFO:teuthology.orchestra.run.trial040.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 19 MB/s | 54 MB 00:02 2026-02-22T23:19:19.864 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:19.864 INFO:teuthology.orchestra.run.trial040.stdout:Total 19 MB/s | 54 MB 00:02 2026-02-22T23:19:19.864 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:19.866 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:19.866 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:20.018 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:20.042 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:20.042 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:20.076 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:20.076 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:20.077 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:20.077 INFO:teuthology.orchestra.run.trial006.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:20.077 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:20.077 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:20.133 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install ceph-fuse-debuginfo 2026-02-22T23:19:20.449 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:16 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:20.533 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:20.534 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:20.534 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:20.534 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 6.8 M 2026-02-22T23:19:20.534 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 36 M 2026-02-22T23:19:20.534 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:20.754 INFO:teuthology.orchestra.run.trial006.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 31 MB/s | 6.8 MB 00:00 2026-02-22T23:19:20.755 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:20.755 INFO:teuthology.orchestra.run.trial006.stdout:Total 31 MB/s | 6.8 MB 00:00 2026-02-22T23:19:20.755 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:20.756 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:20.756 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:20.794 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:20.794 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:21.206 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:21.224 INFO:teuthology.orchestra.run.trial006.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:21.310 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:21.371 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:21.371 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:21.371 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:21.371 INFO:teuthology.orchestra.run.trial006.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:21.371 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:21.371 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:21.398 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados-devel-debuginfo 2026-02-22T23:19:21.712 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:17 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 112 k 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 493 k 2026-02-22T23:19:21.797 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:21.884 INFO:teuthology.orchestra.run.trial006.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 1.3 MB/s | 112 kB 00:00 2026-02-22T23:19:21.885 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:21.885 INFO:teuthology.orchestra.run.trial006.stdout:Total 1.2 MB/s | 112 kB 00:00 2026-02-22T23:19:21.885 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:21.886 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:21.886 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:21.904 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:21.904 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:21.944 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:21.961 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:22.018 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:22.076 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:22.076 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:22.076 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:22.076 INFO:teuthology.orchestra.run.trial006.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:22.076 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:22.076 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:22.103 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install libcephfs2-debuginfo 2026-02-22T23:19:22.417 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:18 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:22.502 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:22.503 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:22.504 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 6.0 M 2026-02-22T23:19:22.504 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 31 M 2026-02-22T23:19:22.504 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:22.706 INFO:teuthology.orchestra.run.trial006.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 29 MB/s | 6.0 MB 00:00 2026-02-22T23:19:22.706 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:22.707 INFO:teuthology.orchestra.run.trial006.stdout:Total 29 MB/s | 6.0 MB 00:00 2026-02-22T23:19:22.707 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:22.708 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:22.709 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:22.744 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:22.744 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:22.821 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:22.839 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:23.090 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:23.107 INFO:teuthology.orchestra.run.trial006.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:23.117 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:23.135 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:23.158 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:23.192 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:23.192 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:23.209 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:23.216 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:23.216 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:23.216 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:23.216 INFO:teuthology.orchestra.run.trial080.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:23.217 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:23.217 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:23.248 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-cephfs-debuginfo 2026-02-22T23:19:23.252 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:23.252 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:23.252 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:23.252 INFO:teuthology.orchestra.run.trial006.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:23.252 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:23.252 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:23.280 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librados2-debuginfo 2026-02-22T23:19:23.509 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:23.559 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:51 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:23.567 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:23.567 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:23.567 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:23.567 INFO:teuthology.orchestra.run.trial040.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:23.567 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:23.568 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:23.592 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:19 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:23.598 DEBUG:teuthology.orchestra.run.trial040:> sudo yum -y install rbd-nbd-debuginfo 2026-02-22T23:19:23.643 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 628 k 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 2.1 M 2026-02-22T23:19:23.644 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:23.675 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 47 M 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 302 M 2026-02-22T23:19:23.676 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:23.754 INFO:teuthology.orchestra.run.trial080.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 5.6 MB/s | 628 kB 00:00 2026-02-22T23:19:23.754 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:23.755 INFO:teuthology.orchestra.run.trial080.stdout:Total 5.6 MB/s | 628 kB 00:00 2026-02-22T23:19:23.755 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:23.756 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:23.756 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:23.777 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:23.777 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:23.843 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:23.860 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:23.908 INFO:teuthology.orchestra.run.trial040.stdout:Last metadata expiration check: 0:04:30 ago on Sun 22 Feb 2026 11:14:53 PM UTC. 2026-02-22T23:19:23.917 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:23.975 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:23.975 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:23.975 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:23.975 INFO:teuthology.orchestra.run.trial080.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:23.975 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:23.975 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:23.990 INFO:teuthology.orchestra.run.trial040.stdout:Dependencies resolved. 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout: Package Arch Version Repo Size 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout:Installing: 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout:Transaction Summary 2026-02-22T23:19:23.991 INFO:teuthology.orchestra.run.trial040.stdout:================================================================================ 2026-02-22T23:19:23.992 INFO:teuthology.orchestra.run.trial040.stdout:Install 1 Package 2026-02-22T23:19:23.992 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:23.992 INFO:teuthology.orchestra.run.trial040.stdout:Total download size: 1.4 M 2026-02-22T23:19:23.992 INFO:teuthology.orchestra.run.trial040.stdout:Installed size: 6.6 M 2026-02-22T23:19:23.992 INFO:teuthology.orchestra.run.trial040.stdout:Downloading Packages: 2026-02-22T23:19:24.002 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rados-debuginfo 2026-02-22T23:19:24.182 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:24.240 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:24.240 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:24.241 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:24.241 INFO:teuthology.orchestra.run.trial189.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:24.241 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:24.241 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:24.296 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install ceph-fuse-debuginfo 2026-02-22T23:19:24.317 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:52 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:24.401 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:24.401 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:24.401 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:24.401 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:24.401 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:24.401 INFO:teuthology.orchestra.run.trial080.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 779 k 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 2.8 M 2026-02-22T23:19:24.402 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:24.534 INFO:teuthology.orchestra.run.trial040.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 2.5 MB/s | 1.4 MB 00:00 2026-02-22T23:19:24.534 INFO:teuthology.orchestra.run.trial040.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:24.535 INFO:teuthology.orchestra.run.trial040.stdout:Total 2.5 MB/s | 1.4 MB 00:00 2026-02-22T23:19:24.535 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction check 2026-02-22T23:19:24.536 INFO:teuthology.orchestra.run.trial040.stdout:Transaction check succeeded. 2026-02-22T23:19:24.536 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction test 2026-02-22T23:19:24.559 INFO:teuthology.orchestra.run.trial040.stdout:Transaction test succeeded. 2026-02-22T23:19:24.559 INFO:teuthology.orchestra.run.trial040.stdout:Running transaction 2026-02-22T23:19:24.608 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:40 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:24.662 INFO:teuthology.orchestra.run.trial080.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 2.9 MB/s | 779 kB 00:00 2026-02-22T23:19:24.663 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:24.663 INFO:teuthology.orchestra.run.trial040.stdout: Preparing : 1/1 2026-02-22T23:19:24.664 INFO:teuthology.orchestra.run.trial080.stdout:Total 2.9 MB/s | 779 kB 00:00 2026-02-22T23:19:24.664 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:24.665 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:24.665 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:24.677 INFO:teuthology.orchestra.run.trial006.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 47 MB/s | 47 MB 00:01 2026-02-22T23:19:24.677 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:24.677 INFO:teuthology.orchestra.run.trial006.stdout:Total 47 MB/s | 47 MB 00:01 2026-02-22T23:19:24.678 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:24.680 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:24.680 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:24.681 INFO:teuthology.orchestra.run.trial040.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:24.685 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:24.686 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:24.690 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:24.690 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:24.691 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:24.692 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 6.8 M 2026-02-22T23:19:24.692 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 36 M 2026-02-22T23:19:24.692 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:24.742 INFO:teuthology.orchestra.run.trial040.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:24.757 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:24.774 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:24.800 INFO:teuthology.orchestra.run.trial040.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:24.800 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:24.801 INFO:teuthology.orchestra.run.trial040.stdout:Installed: 2026-02-22T23:19:24.801 INFO:teuthology.orchestra.run.trial040.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:24.801 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:24.801 INFO:teuthology.orchestra.run.trial040.stdout:Complete! 2026-02-22T23:19:24.830 DEBUG:teuthology.parallel:result is None 2026-02-22T23:19:24.831 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:24.837 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:24.838 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:24.889 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:24.889 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:24.889 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:24.889 INFO:teuthology.orchestra.run.trial080.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:24.889 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:24.889 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:24.911 INFO:teuthology.orchestra.run.trial189.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 31 MB/s | 6.8 MB 00:00 2026-02-22T23:19:24.911 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:24.912 INFO:teuthology.orchestra.run.trial189.stdout:Total 31 MB/s | 6.8 MB 00:00 2026-02-22T23:19:24.912 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:24.913 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:24.913 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:24.914 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rbd-debuginfo 2026-02-22T23:19:24.951 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:24.951 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:25.220 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:53 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:25.302 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 1.1 M 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 4.3 M 2026-02-22T23:19:25.303 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:25.343 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:25.360 INFO:teuthology.orchestra.run.trial189.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:25.424 INFO:teuthology.orchestra.run.trial080.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 9.4 MB/s | 1.1 MB 00:00 2026-02-22T23:19:25.424 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:25.424 INFO:teuthology.orchestra.run.trial080.stdout:Total 9.4 MB/s | 1.1 MB 00:00 2026-02-22T23:19:25.425 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:25.426 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:25.426 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:25.447 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:25.447 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:25.450 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:25.509 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:25.509 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:25.509 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:25.509 INFO:teuthology.orchestra.run.trial189.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:25.509 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:25.509 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:25.537 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados-devel-debuginfo 2026-02-22T23:19:25.538 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:25.555 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:25.613 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:25.671 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:25.672 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:25.672 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:25.672 INFO:teuthology.orchestra.run.trial080.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:25.672 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:25.672 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:25.699 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rgw-debuginfo 2026-02-22T23:19:25.850 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:41 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:25.932 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:25.933 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:25.934 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 112 k 2026-02-22T23:19:25.934 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 493 k 2026-02-22T23:19:25.934 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:26.004 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:54 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:26.019 INFO:teuthology.orchestra.run.trial189.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 1.3 MB/s | 112 kB 00:00 2026-02-22T23:19:26.019 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:26.019 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.3 MB/s | 112 kB 00:00 2026-02-22T23:19:26.019 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:26.021 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:26.021 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:26.039 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:26.039 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:26.079 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:26.087 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:26.088 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 339 k 2026-02-22T23:19:26.088 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 1.0 M 2026-02-22T23:19:26.088 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:26.095 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:26.152 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:26.187 INFO:teuthology.orchestra.run.trial080.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 3.3 MB/s | 339 kB 00:00 2026-02-22T23:19:26.188 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:26.188 INFO:teuthology.orchestra.run.trial080.stdout:Total 3.3 MB/s | 339 kB 00:00 2026-02-22T23:19:26.188 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:26.189 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:26.190 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:26.209 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:26.209 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:26.210 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:26.210 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:26.210 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:26.211 INFO:teuthology.orchestra.run.trial189.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:26.211 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:26.211 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:26.236 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install libcephfs2-debuginfo 2026-02-22T23:19:26.260 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:26.277 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:26.334 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:26.392 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:26.392 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:26.392 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:26.392 INFO:teuthology.orchestra.run.trial080.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:26.392 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:26.392 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:26.418 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-fuse-debuginfo 2026-02-22T23:19:26.549 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:42 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:26.631 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:26.631 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:26.632 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:26.633 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 6.0 M 2026-02-22T23:19:26.633 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 31 M 2026-02-22T23:19:26.633 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:26.724 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:54 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 547 k 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 3.0 M 2026-02-22T23:19:26.808 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:26.838 INFO:teuthology.orchestra.run.trial189.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 29 MB/s | 6.0 MB 00:00 2026-02-22T23:19:26.839 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:26.839 INFO:teuthology.orchestra.run.trial189.stdout:Total 29 MB/s | 6.0 MB 00:00 2026-02-22T23:19:26.839 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:26.840 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:26.841 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:26.876 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:26.877 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:26.930 INFO:teuthology.orchestra.run.trial080.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 4.4 MB/s | 547 kB 00:00 2026-02-22T23:19:26.930 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:26.931 INFO:teuthology.orchestra.run.trial080.stdout:Total 4.4 MB/s | 547 kB 00:00 2026-02-22T23:19:26.931 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:26.932 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:26.932 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:26.951 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:26.951 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:27.018 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:27.035 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:27.092 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:27.150 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:27.150 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:27.150 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:27.150 INFO:teuthology.orchestra.run.trial080.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:27.150 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:27.150 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:27.176 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-mirror-debuginfo 2026-02-22T23:19:27.221 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:27.240 INFO:teuthology.orchestra.run.trial189.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:27.322 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:27.380 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:27.381 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:27.381 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:27.381 INFO:teuthology.orchestra.run.trial189.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:27.381 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:27.381 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:27.407 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librados2-debuginfo 2026-02-22T23:19:27.475 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:27.485 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:04:55 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:27.493 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:27.569 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:27.569 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:27.569 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:27.569 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 54 M 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 353 M 2026-02-22T23:19:27.570 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:27.723 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:43 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:27.747 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:27.805 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:27.806 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:27.807 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:27.807 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:27.807 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 47 M 2026-02-22T23:19:27.807 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 302 M 2026-02-22T23:19:27.807 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:27.807 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:27.807 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:27.808 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:27.808 INFO:teuthology.orchestra.run.trial006.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:27.808 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:27.808 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:27.838 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install librbd1-debuginfo 2026-02-22T23:19:28.150 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:24 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:28.233 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 55 M 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 374 M 2026-02-22T23:19:28.234 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:28.828 INFO:teuthology.orchestra.run.trial080.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 43 MB/s | 54 MB 00:01 2026-02-22T23:19:28.829 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:28.829 INFO:teuthology.orchestra.run.trial080.stdout:Total 43 MB/s | 54 MB 00:01 2026-02-22T23:19:28.830 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:28.831 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:28.832 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:29.006 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:29.006 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:29.655 INFO:teuthology.orchestra.run.trial189.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 26 MB/s | 47 MB 00:01 2026-02-22T23:19:29.656 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:29.656 INFO:teuthology.orchestra.run.trial189.stdout:Total 25 MB/s | 47 MB 00:01 2026-02-22T23:19:29.656 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:29.657 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:29.657 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:29.816 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:29.816 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:30.403 INFO:teuthology.orchestra.run.trial006.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 25 MB/s | 55 MB 00:02 2026-02-22T23:19:30.404 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:30.405 INFO:teuthology.orchestra.run.trial006.stdout:Total 25 MB/s | 55 MB 00:02 2026-02-22T23:19:30.405 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:30.406 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:30.406 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:30.586 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:30.587 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:32.160 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:32.178 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:32.463 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:32.480 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:32.492 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:32.550 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:32.550 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:32.550 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:32.550 INFO:teuthology.orchestra.run.trial080.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:32.550 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:32.550 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:32.581 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-nbd-debuginfo 2026-02-22T23:19:32.737 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:32.797 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-22T23:19:32.797 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:32.797 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:32.797 INFO:teuthology.orchestra.run.trial189.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:32.797 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:32.797 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:32.829 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install librbd1-debuginfo 2026-02-22T23:19:32.896 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:05:00 ago on Sun 22 Feb 2026 11:14:32 PM UTC. 2026-02-22T23:19:32.979 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-22T23:19:32.979 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-22T23:19:32.980 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:32.981 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 1.4 M 2026-02-22T23:19:32.981 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 6.6 M 2026-02-22T23:19:32.981 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-22T23:19:33.103 INFO:teuthology.orchestra.run.trial080.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 11 MB/s | 1.4 MB 00:00 2026-02-22T23:19:33.104 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:33.104 INFO:teuthology.orchestra.run.trial080.stdout:Total 11 MB/s | 1.4 MB 00:00 2026-02-22T23:19:33.104 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-22T23:19:33.105 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-22T23:19:33.105 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-22T23:19:33.128 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-22T23:19:33.128 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-22T23:19:33.137 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:49 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:33.220 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:33.220 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:33.220 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:33.220 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:33.220 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:33.220 INFO:teuthology.orchestra.run.trial189.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 55 M 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 374 M 2026-02-22T23:19:33.221 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:33.240 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-22T23:19:33.257 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:33.317 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:33.375 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:33.375 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:33.376 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-22T23:19:33.376 INFO:teuthology.orchestra.run.trial080.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:33.376 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:33.376 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-22T23:19:33.405 DEBUG:teuthology.parallel:result is None 2026-02-22T23:19:33.721 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:33.739 INFO:teuthology.orchestra.run.trial006.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:34.040 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:34.098 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:34.098 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:34.098 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:34.099 INFO:teuthology.orchestra.run.trial006.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:34.099 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:34.099 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:34.129 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-cephfs-debuginfo 2026-02-22T23:19:34.324 INFO:teuthology.orchestra.run.trial189.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 49 MB/s | 55 MB 00:01 2026-02-22T23:19:34.325 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:34.325 INFO:teuthology.orchestra.run.trial189.stdout:Total 49 MB/s | 55 MB 00:01 2026-02-22T23:19:34.325 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:34.327 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:34.327 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:34.438 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:30 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:34.506 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:34.507 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:34.521 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 628 k 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 2.1 M 2026-02-22T23:19:34.522 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:34.631 INFO:teuthology.orchestra.run.trial006.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 5.6 MB/s | 628 kB 00:00 2026-02-22T23:19:34.632 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:34.632 INFO:teuthology.orchestra.run.trial006.stdout:Total 5.6 MB/s | 628 kB 00:00 2026-02-22T23:19:34.633 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:34.634 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:34.634 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:34.655 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:34.655 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:34.720 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:34.737 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:34.794 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:34.852 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:34.852 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:34.853 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:34.853 INFO:teuthology.orchestra.run.trial006.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:34.853 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:34.853 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:34.879 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rados-debuginfo 2026-02-22T23:19:35.194 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:31 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:35.278 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:35.278 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:35.279 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 779 k 2026-02-22T23:19:35.280 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 2.8 M 2026-02-22T23:19:35.280 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:35.391 INFO:teuthology.orchestra.run.trial006.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 6.8 MB/s | 779 kB 00:00 2026-02-22T23:19:35.391 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:35.392 INFO:teuthology.orchestra.run.trial006.stdout:Total 6.7 MB/s | 779 kB 00:00 2026-02-22T23:19:35.392 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:35.393 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:35.393 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:35.413 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:35.413 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:35.485 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:35.502 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:35.560 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:35.619 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:35.619 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:35.619 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:35.619 INFO:teuthology.orchestra.run.trial006.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:35.619 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:35.619 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:35.645 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rbd-debuginfo 2026-02-22T23:19:35.956 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:31 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:36.039 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 1.1 M 2026-02-22T23:19:36.040 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 4.3 M 2026-02-22T23:19:36.041 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:36.162 INFO:teuthology.orchestra.run.trial006.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 9.3 MB/s | 1.1 MB 00:00 2026-02-22T23:19:36.162 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:36.162 INFO:teuthology.orchestra.run.trial006.stdout:Total 9.3 MB/s | 1.1 MB 00:00 2026-02-22T23:19:36.162 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:36.164 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:36.164 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:36.185 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:36.185 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:36.275 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:36.292 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:36.349 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:36.408 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:36.408 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:36.408 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:36.408 INFO:teuthology.orchestra.run.trial006.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:36.408 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:36.409 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:36.434 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install python3-rgw-debuginfo 2026-02-22T23:19:36.748 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:32 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:36.834 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:36.834 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:36.834 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:36.834 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 339 k 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 1.0 M 2026-02-22T23:19:36.835 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:36.933 INFO:teuthology.orchestra.run.trial006.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 3.4 MB/s | 339 kB 00:00 2026-02-22T23:19:36.933 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:36.933 INFO:teuthology.orchestra.run.trial006.stdout:Total 3.3 MB/s | 339 kB 00:00 2026-02-22T23:19:36.934 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:36.935 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:36.935 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:36.954 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:36.954 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:37.003 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:37.021 INFO:teuthology.orchestra.run.trial006.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:37.077 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:37.136 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:37.136 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:37.136 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:37.136 INFO:teuthology.orchestra.run.trial006.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:37.136 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:37.137 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:37.163 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-fuse-debuginfo 2026-02-22T23:19:37.478 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:33 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:37.561 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:37.562 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:37.563 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 547 k 2026-02-22T23:19:37.563 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 3.0 M 2026-02-22T23:19:37.563 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:37.672 INFO:teuthology.orchestra.run.trial006.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 4.9 MB/s | 547 kB 00:00 2026-02-22T23:19:37.672 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:37.672 INFO:teuthology.orchestra.run.trial006.stdout:Total 4.8 MB/s | 547 kB 00:00 2026-02-22T23:19:37.672 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:37.674 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:37.674 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:37.694 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:37.694 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:37.703 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:37.722 INFO:teuthology.orchestra.run.trial189.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:37.757 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:37.775 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:37.836 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:37.896 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:37.896 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:37.896 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:37.897 INFO:teuthology.orchestra.run.trial006.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:37.897 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:37.897 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:37.926 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-mirror-debuginfo 2026-02-22T23:19:38.032 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:38.092 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:38.092 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:38.092 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:38.092 INFO:teuthology.orchestra.run.trial189.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:38.092 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:38.092 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:38.128 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-cephfs-debuginfo 2026-02-22T23:19:38.243 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:34 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:38.327 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:38.327 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:38.328 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:38.328 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:38.328 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:38.328 INFO:teuthology.orchestra.run.trial006.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-22T23:19:38.328 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:38.328 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:38.328 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:38.329 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:38.329 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:38.329 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 54 M 2026-02-22T23:19:38.329 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 353 M 2026-02-22T23:19:38.329 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:38.443 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:54 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:38.525 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:38.525 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:38.525 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:38.525 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 628 k 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 2.1 M 2026-02-22T23:19:38.526 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:39.019 INFO:teuthology.orchestra.run.trial189.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 1.2 MB/s | 628 kB 00:00 2026-02-22T23:19:39.019 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:39.019 INFO:teuthology.orchestra.run.trial189.stdout:Total 1.2 MB/s | 628 kB 00:00 2026-02-22T23:19:39.020 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:39.021 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:39.021 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:39.041 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:39.041 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:39.105 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:39.122 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:39.180 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:39.239 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-22T23:19:39.239 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:39.239 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:39.239 INFO:teuthology.orchestra.run.trial189.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:39.239 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:39.239 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:39.264 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rados-debuginfo 2026-02-22T23:19:39.424 INFO:teuthology.orchestra.run.trial006.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 49 MB/s | 54 MB 00:01 2026-02-22T23:19:39.424 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:39.425 INFO:teuthology.orchestra.run.trial006.stdout:Total 49 MB/s | 54 MB 00:01 2026-02-22T23:19:39.425 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:39.426 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:39.426 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:39.576 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:55 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:39.601 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:39.601 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:39.658 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:39.659 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:39.659 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:39.659 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:39.659 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:39.659 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-22T23:19:39.659 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:39.660 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:39.660 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:39.660 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:39.660 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:39.660 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 779 k 2026-02-22T23:19:39.660 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 2.8 M 2026-02-22T23:19:39.660 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:39.768 INFO:teuthology.orchestra.run.trial189.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 6.9 MB/s | 779 kB 00:00 2026-02-22T23:19:39.769 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:39.769 INFO:teuthology.orchestra.run.trial189.stdout:Total 6.9 MB/s | 779 kB 00:00 2026-02-22T23:19:39.770 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:39.771 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:39.771 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:39.791 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:39.791 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:39.862 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:39.879 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:39.936 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:39.995 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-22T23:19:39.995 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:39.995 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:39.995 INFO:teuthology.orchestra.run.trial189.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:39.995 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:39.995 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:40.021 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rbd-debuginfo 2026-02-22T23:19:40.331 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:56 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:40.414 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:40.415 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:40.416 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:40.416 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:40.416 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 1.1 M 2026-02-22T23:19:40.416 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 4.3 M 2026-02-22T23:19:40.416 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:40.536 INFO:teuthology.orchestra.run.trial189.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 9.4 MB/s | 1.1 MB 00:00 2026-02-22T23:19:40.537 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:40.537 INFO:teuthology.orchestra.run.trial189.stdout:Total 9.4 MB/s | 1.1 MB 00:00 2026-02-22T23:19:40.537 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:40.538 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:40.539 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:40.560 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:40.560 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:40.652 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:40.669 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:40.728 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:40.787 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:40.787 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:40.787 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:40.787 INFO:teuthology.orchestra.run.trial189.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:40.787 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:40.787 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:40.814 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install python3-rgw-debuginfo 2026-02-22T23:19:41.129 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:57 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:41.212 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:41.212 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:41.212 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:41.212 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:41.212 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:41.212 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-22T23:19:41.212 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:41.213 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:41.213 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:41.213 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:41.213 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:41.213 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 339 k 2026-02-22T23:19:41.213 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 1.0 M 2026-02-22T23:19:41.213 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:41.311 INFO:teuthology.orchestra.run.trial189.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 3.4 MB/s | 339 kB 00:00 2026-02-22T23:19:41.312 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:41.312 INFO:teuthology.orchestra.run.trial189.stdout:Total 3.3 MB/s | 339 kB 00:00 2026-02-22T23:19:41.312 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:41.313 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:41.313 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:41.332 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:41.332 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:41.382 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:41.399 INFO:teuthology.orchestra.run.trial189.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:41.455 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:41.514 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-22T23:19:41.514 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:41.514 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:41.514 INFO:teuthology.orchestra.run.trial189.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:41.514 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:41.514 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:41.540 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-fuse-debuginfo 2026-02-22T23:19:41.853 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:57 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:41.935 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 547 k 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 3.0 M 2026-02-22T23:19:41.936 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:42.045 INFO:teuthology.orchestra.run.trial189.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 4.9 MB/s | 547 kB 00:00 2026-02-22T23:19:42.046 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:42.046 INFO:teuthology.orchestra.run.trial189.stdout:Total 4.9 MB/s | 547 kB 00:00 2026-02-22T23:19:42.046 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:42.048 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:42.048 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:42.067 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:42.068 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:42.132 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:42.149 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:42.213 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:42.273 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:42.273 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:42.273 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:42.274 INFO:teuthology.orchestra.run.trial189.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:42.274 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:42.274 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:42.304 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-mirror-debuginfo 2026-02-22T23:19:42.616 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:04:58 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:42.698 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:42.699 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:42.699 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:42.699 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:42.699 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:42.699 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-22T23:19:42.699 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:42.700 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:42.700 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:42.700 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:42.700 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:42.700 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 54 M 2026-02-22T23:19:42.700 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 353 M 2026-02-22T23:19:42.700 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:42.803 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:42.821 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:43.146 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:43.205 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:43.205 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:43.205 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:43.205 INFO:teuthology.orchestra.run.trial006.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:43.205 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:43.206 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:43.238 DEBUG:teuthology.orchestra.run.trial006:> sudo yum -y install rbd-nbd-debuginfo 2026-02-22T23:19:43.551 INFO:teuthology.orchestra.run.trial006.stdout:Last metadata expiration check: 0:04:39 ago on Sun 22 Feb 2026 11:15:04 PM UTC. 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout:Dependencies resolved. 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout: Package Arch Version Repo Size 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout:Installing: 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout:Transaction Summary 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout:================================================================================ 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout:Install 1 Package 2026-02-22T23:19:43.635 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:43.636 INFO:teuthology.orchestra.run.trial006.stdout:Total download size: 1.4 M 2026-02-22T23:19:43.636 INFO:teuthology.orchestra.run.trial006.stdout:Installed size: 6.6 M 2026-02-22T23:19:43.636 INFO:teuthology.orchestra.run.trial006.stdout:Downloading Packages: 2026-02-22T23:19:43.787 INFO:teuthology.orchestra.run.trial189.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 49 MB/s | 54 MB 00:01 2026-02-22T23:19:43.788 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:43.788 INFO:teuthology.orchestra.run.trial189.stdout:Total 49 MB/s | 54 MB 00:01 2026-02-22T23:19:43.789 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:43.790 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:43.790 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:43.806 INFO:teuthology.orchestra.run.trial006.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 8.0 MB/s | 1.4 MB 00:00 2026-02-22T23:19:43.807 INFO:teuthology.orchestra.run.trial006.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:43.807 INFO:teuthology.orchestra.run.trial006.stdout:Total 7.9 MB/s | 1.4 MB 00:00 2026-02-22T23:19:43.807 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction check 2026-02-22T23:19:43.808 INFO:teuthology.orchestra.run.trial006.stdout:Transaction check succeeded. 2026-02-22T23:19:43.809 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction test 2026-02-22T23:19:43.832 INFO:teuthology.orchestra.run.trial006.stdout:Transaction test succeeded. 2026-02-22T23:19:43.832 INFO:teuthology.orchestra.run.trial006.stdout:Running transaction 2026-02-22T23:19:43.936 INFO:teuthology.orchestra.run.trial006.stdout: Preparing : 1/1 2026-02-22T23:19:43.954 INFO:teuthology.orchestra.run.trial006.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:43.964 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:43.965 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:44.014 INFO:teuthology.orchestra.run.trial006.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:44.074 INFO:teuthology.orchestra.run.trial006.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:44.074 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:44.075 INFO:teuthology.orchestra.run.trial006.stdout:Installed: 2026-02-22T23:19:44.075 INFO:teuthology.orchestra.run.trial006.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:44.075 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:44.075 INFO:teuthology.orchestra.run.trial006.stdout:Complete! 2026-02-22T23:19:44.106 DEBUG:teuthology.parallel:result is None 2026-02-22T23:19:47.018 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:47.035 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:47.355 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:47.414 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-22T23:19:47.414 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:47.414 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:47.414 INFO:teuthology.orchestra.run.trial189.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:47.414 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:47.414 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:47.446 DEBUG:teuthology.orchestra.run.trial189:> sudo yum -y install rbd-nbd-debuginfo 2026-02-22T23:19:47.756 INFO:teuthology.orchestra.run.trial189.stdout:Last metadata expiration check: 0:05:03 ago on Sun 22 Feb 2026 11:14:44 PM UTC. 2026-02-22T23:19:47.839 INFO:teuthology.orchestra.run.trial189.stdout:Dependencies resolved. 2026-02-22T23:19:47.839 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:47.839 INFO:teuthology.orchestra.run.trial189.stdout: Package Arch Version Repo Size 2026-02-22T23:19:47.839 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:47.839 INFO:teuthology.orchestra.run.trial189.stdout:Installing: 2026-02-22T23:19:47.839 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-22T23:19:47.839 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:47.840 INFO:teuthology.orchestra.run.trial189.stdout:Transaction Summary 2026-02-22T23:19:47.840 INFO:teuthology.orchestra.run.trial189.stdout:================================================================================ 2026-02-22T23:19:47.840 INFO:teuthology.orchestra.run.trial189.stdout:Install 1 Package 2026-02-22T23:19:47.840 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:47.840 INFO:teuthology.orchestra.run.trial189.stdout:Total download size: 1.4 M 2026-02-22T23:19:47.840 INFO:teuthology.orchestra.run.trial189.stdout:Installed size: 6.6 M 2026-02-22T23:19:47.840 INFO:teuthology.orchestra.run.trial189.stdout:Downloading Packages: 2026-02-22T23:19:47.962 INFO:teuthology.orchestra.run.trial189.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 11 MB/s | 1.4 MB 00:00 2026-02-22T23:19:47.963 INFO:teuthology.orchestra.run.trial189.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:19:47.963 INFO:teuthology.orchestra.run.trial189.stdout:Total 11 MB/s | 1.4 MB 00:00 2026-02-22T23:19:47.963 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction check 2026-02-22T23:19:47.965 INFO:teuthology.orchestra.run.trial189.stdout:Transaction check succeeded. 2026-02-22T23:19:47.965 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction test 2026-02-22T23:19:47.988 INFO:teuthology.orchestra.run.trial189.stdout:Transaction test succeeded. 2026-02-22T23:19:47.988 INFO:teuthology.orchestra.run.trial189.stdout:Running transaction 2026-02-22T23:19:48.093 INFO:teuthology.orchestra.run.trial189.stdout: Preparing : 1/1 2026-02-22T23:19:48.110 INFO:teuthology.orchestra.run.trial189.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:48.170 INFO:teuthology.orchestra.run.trial189.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:48.230 INFO:teuthology.orchestra.run.trial189.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-22T23:19:48.230 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:48.230 INFO:teuthology.orchestra.run.trial189.stdout:Installed: 2026-02-22T23:19:48.230 INFO:teuthology.orchestra.run.trial189.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:19:48.230 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:48.230 INFO:teuthology.orchestra.run.trial189.stdout:Complete! 2026-02-22T23:19:48.261 DEBUG:teuthology.parallel:result is None 2026-02-22T23:19:48.262 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:19:48.430 DEBUG:teuthology.orchestra.run.trial006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-22T23:19:48.451 INFO:teuthology.orchestra.run.trial006.stdout:19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.452 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.452 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-22T23:19:48.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:19:48.591 DEBUG:teuthology.orchestra.run.trial040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-22T23:19:48.611 INFO:teuthology.orchestra.run.trial040.stdout:19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.612 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.612 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-22T23:19:48.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:19:48.756 DEBUG:teuthology.orchestra.run.trial080:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-22T23:19:48.777 INFO:teuthology.orchestra.run.trial080.stdout:19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.777 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.777 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-22T23:19:48.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:19:48.898 DEBUG:teuthology.orchestra.run.trial189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-22T23:19:48.925 INFO:teuthology.orchestra.run.trial189.stdout:19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.926 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-22T23:19:48.926 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-22T23:19:48.929 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-22T23:19:48.929 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:48.929 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T23:19:48.959 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:19:48.959 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T23:19:48.987 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:19:48.987 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T23:19:49.013 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:19:49.013 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T23:19:49.040 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-22T23:19:49.040 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:49.040 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T23:19:49.065 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T23:19:49.129 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:19:49.130 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T23:19:49.155 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T23:19:49.219 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:19:49.219 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T23:19:49.245 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T23:19:49.309 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:19:49.309 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T23:19:49.335 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T23:19:49.399 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-22T23:19:49.400 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:49.400 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T23:19:49.425 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T23:19:49.488 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:19:49.489 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T23:19:49.514 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T23:19:49.581 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:19:49.581 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T23:19:49.606 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T23:19:49.670 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:19:49.670 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T23:19:49.695 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T23:19:49.759 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-22T23:19:49.759 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:49.760 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T23:19:49.785 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T23:19:49.849 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:19:49.849 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T23:19:49.874 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T23:19:49.938 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:19:49.938 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T23:19:49.964 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T23:19:50.027 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:19:50.027 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T23:19:50.052 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T23:19:50.116 INFO:teuthology.run_tasks:Running task ceph... 2026-02-22T23:19:50.196 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-22T23:19:50.196 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 777 /var/log/ceph 2026-02-22T23:19:50.199 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod 777 /var/log/ceph 2026-02-22T23:19:50.201 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 777 /var/log/ceph 2026-02-22T23:19:50.203 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod 777 /var/log/ceph 2026-02-22T23:19:50.226 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-22T23:19:50.226 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T23:19:50.262 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T23:19:50.265 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T23:19:50.267 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T23:19:50.290 INFO:tasks.ceph:Creating extra log directories... 2026-02-22T23:19:50.290 DEBUG:teuthology.orchestra.run.trial006:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T23:19:50.326 DEBUG:teuthology.orchestra.run.trial040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T23:19:50.330 DEBUG:teuthology.orchestra.run.trial080:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T23:19:50.332 DEBUG:teuthology.orchestra.run.trial189:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T23:19:50.376 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-22T23:19:50.377 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-22T23:19:50.377 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361', 'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'description': 'rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '64361', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'reached quota', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(SLOW_OPS\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(OBJECT_MISPLACED\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request', 'OSD bench result'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'stop-daemons-timeout': 1200, 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 5907, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'targets': {'trial006.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHtq4id0Rnt4Dlue3y07cxPgX+ypWyWEsAZ6LPQ+y660ESLNEaR1PP/DxoDSDNGaneZOP6cNPTyatKww8WF16q0=', 'trial040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXHopXURMt5cdZ5VAgzVR3xz7qzxurn1Fy31EK4ll2icR0n21w2nPGqlQYz6wk1ecv63AfkJEgCUyPI26uxGYQ=', 'trial080.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOHh1jaXgsyBoMa05cVAm3f5zIYMzDtEBvSW3S2p6YZju4hFoH+f5Jdwr0/ap6UVdGqNtsyxijREtEAE1RnSKK4=', 'trial189.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOzk7PS1aiuAjw0M5FrfAHxbpnMP1CuoP60ehFTcObSPhZrIWiSvzG1VMUIobglYgYW/1bKvxZoJgTZmzskRsAc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rados/test.sh']}, 'env': {'ALLOW_TIMEOUTS': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["-- Don't run this test on tiny clusters\nif py_len(yaml.roles) < 4 then reject() end\n"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-22_19:35:40', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653'} 2026-02-22T23:19:50.378 DEBUG:teuthology.orchestra.run.trial006:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T23:19:50.391 DEBUG:teuthology.orchestra.run.trial040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T23:19:50.395 DEBUG:teuthology.orchestra.run.trial080:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T23:19:50.398 DEBUG:teuthology.orchestra.run.trial189:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T23:19:50.415 DEBUG:teuthology.orchestra.run.trial006:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T23:19:50.448 DEBUG:teuthology.orchestra.run.trial040:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T23:19:50.452 DEBUG:teuthology.orchestra.run.trial080:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T23:19:50.453 DEBUG:teuthology.orchestra.run.trial189:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T23:19:50.478 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:50.478 DEBUG:teuthology.orchestra.run.trial006:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T23:19:50.526 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T23:19:50.526 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_1 2026-02-22T23:19:50.583 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T23:19:50.583 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:50.583 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1296 Links: 1 2026-02-22T23:19:50.583 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:50.583 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:50.583 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-22 23:15:48.837024949 +0000 2026-02-22T23:19:50.584 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-22 23:15:10.296783025 +0000 2026-02-22T23:19:50.584 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-22 23:15:10.296783025 +0000 2026-02-22T23:19:50.584 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-22 23:15:10.296783025 +0000 2026-02-22T23:19:50.584 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T23:19:50.646 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-22T23:19:50.646 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-22T23:19:50.646 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000178351 s, 2.9 MB/s 2026-02-22T23:19:50.647 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T23:19:50.704 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_2 2026-02-22T23:19:50.762 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1312 Links: 1 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-22 23:15:48.838024928 +0000 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-22 23:15:10.297783007 +0000 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-22 23:15:10.297783007 +0000 2026-02-22T23:19:50.763 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-22 23:15:10.297783007 +0000 2026-02-22T23:19:50.764 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T23:19:50.824 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-22T23:19:50.825 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-22T23:19:50.825 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000142825 s, 3.6 MB/s 2026-02-22T23:19:50.826 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T23:19:50.885 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_3 2026-02-22T23:19:50.942 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T23:19:50.942 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:50.942 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-22T23:19:50.942 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:50.943 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:50.943 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-22 23:15:48.838024928 +0000 2026-02-22T23:19:50.943 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-22 23:15:10.295783044 +0000 2026-02-22T23:19:50.943 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-22 23:15:10.295783044 +0000 2026-02-22T23:19:50.943 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-22 23:15:10.295783044 +0000 2026-02-22T23:19:50.943 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T23:19:51.005 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-22T23:19:51.005 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-22T23:19:51.005 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000143857 s, 3.6 MB/s 2026-02-22T23:19:51.007 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T23:19:51.065 DEBUG:teuthology.orchestra.run.trial006:> stat /dev/vg_nvme/lv_4 2026-02-22T23:19:51.122 INFO:teuthology.orchestra.run.trial006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T23:19:51.122 INFO:teuthology.orchestra.run.trial006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:51.122 INFO:teuthology.orchestra.run.trial006.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-22T23:19:51.122 INFO:teuthology.orchestra.run.trial006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:51.123 INFO:teuthology.orchestra.run.trial006.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:51.123 INFO:teuthology.orchestra.run.trial006.stdout:Access: 2026-02-22 23:15:48.839024909 +0000 2026-02-22T23:19:51.123 INFO:teuthology.orchestra.run.trial006.stdout:Modify: 2026-02-22 23:15:10.296783025 +0000 2026-02-22T23:19:51.123 INFO:teuthology.orchestra.run.trial006.stdout:Change: 2026-02-22 23:15:10.296783025 +0000 2026-02-22T23:19:51.123 INFO:teuthology.orchestra.run.trial006.stdout: Birth: 2026-02-22 23:15:10.296783025 +0000 2026-02-22T23:19:51.123 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T23:19:51.185 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records in 2026-02-22T23:19:51.185 INFO:teuthology.orchestra.run.trial006.stderr:1+0 records out 2026-02-22T23:19:51.185 INFO:teuthology.orchestra.run.trial006.stderr:512 bytes copied, 0.000167221 s, 3.1 MB/s 2026-02-22T23:19:51.186 DEBUG:teuthology.orchestra.run.trial006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T23:19:51.243 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:51.243 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:19:51.244 DEBUG:teuthology.orchestra.run.trial040:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T23:19:51.262 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T23:19:51.262 DEBUG:teuthology.orchestra.run.trial040:> stat /dev/vg_nvme/lv_1 2026-02-22T23:19:51.320 INFO:teuthology.orchestra.run.trial040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T23:19:51.320 INFO:teuthology.orchestra.run.trial040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:51.320 INFO:teuthology.orchestra.run.trial040.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-22T23:19:51.321 INFO:teuthology.orchestra.run.trial040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:51.321 INFO:teuthology.orchestra.run.trial040.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:51.321 INFO:teuthology.orchestra.run.trial040.stdout:Access: 2026-02-22 23:15:36.489834083 +0000 2026-02-22T23:19:51.321 INFO:teuthology.orchestra.run.trial040.stdout:Modify: 2026-02-22 23:14:58.519170350 +0000 2026-02-22T23:19:51.321 INFO:teuthology.orchestra.run.trial040.stdout:Change: 2026-02-22 23:14:58.520170331 +0000 2026-02-22T23:19:51.321 INFO:teuthology.orchestra.run.trial040.stdout: Birth: 2026-02-22 23:14:58.519170350 +0000 2026-02-22T23:19:51.321 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T23:19:51.384 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records in 2026-02-22T23:19:51.384 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records out 2026-02-22T23:19:51.385 INFO:teuthology.orchestra.run.trial040.stderr:512 bytes copied, 0.000162713 s, 3.1 MB/s 2026-02-22T23:19:51.386 DEBUG:teuthology.orchestra.run.trial040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T23:19:51.443 DEBUG:teuthology.orchestra.run.trial040:> stat /dev/vg_nvme/lv_2 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout:Device: 6h/6d Inode: 1302 Links: 1 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout:Access: 2026-02-22 23:15:36.489834083 +0000 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout:Modify: 2026-02-22 23:14:58.520170331 +0000 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout:Change: 2026-02-22 23:14:58.520170331 +0000 2026-02-22T23:19:51.499 INFO:teuthology.orchestra.run.trial040.stdout: Birth: 2026-02-22 23:14:58.520170331 +0000 2026-02-22T23:19:51.499 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T23:19:51.561 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records in 2026-02-22T23:19:51.561 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records out 2026-02-22T23:19:51.561 INFO:teuthology.orchestra.run.trial040.stderr:512 bytes copied, 0.000143086 s, 3.6 MB/s 2026-02-22T23:19:51.562 DEBUG:teuthology.orchestra.run.trial040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T23:19:51.619 DEBUG:teuthology.orchestra.run.trial040:> stat /dev/vg_nvme/lv_3 2026-02-22T23:19:51.675 INFO:teuthology.orchestra.run.trial040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T23:19:51.675 INFO:teuthology.orchestra.run.trial040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:51.675 INFO:teuthology.orchestra.run.trial040.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-22T23:19:51.676 INFO:teuthology.orchestra.run.trial040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:51.676 INFO:teuthology.orchestra.run.trial040.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:51.676 INFO:teuthology.orchestra.run.trial040.stdout:Access: 2026-02-22 23:15:36.489834083 +0000 2026-02-22T23:19:51.676 INFO:teuthology.orchestra.run.trial040.stdout:Modify: 2026-02-22 23:14:58.520170331 +0000 2026-02-22T23:19:51.676 INFO:teuthology.orchestra.run.trial040.stdout:Change: 2026-02-22 23:14:58.520170331 +0000 2026-02-22T23:19:51.676 INFO:teuthology.orchestra.run.trial040.stdout: Birth: 2026-02-22 23:14:58.520170331 +0000 2026-02-22T23:19:51.676 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T23:19:51.739 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records in 2026-02-22T23:19:51.739 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records out 2026-02-22T23:19:51.740 INFO:teuthology.orchestra.run.trial040.stderr:512 bytes copied, 0.000154086 s, 3.3 MB/s 2026-02-22T23:19:51.741 DEBUG:teuthology.orchestra.run.trial040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T23:19:51.798 DEBUG:teuthology.orchestra.run.trial040:> stat /dev/vg_nvme/lv_4 2026-02-22T23:19:51.854 INFO:teuthology.orchestra.run.trial040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout:Device: 6h/6d Inode: 1288 Links: 1 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout:Access: 2026-02-22 23:15:36.490834008 +0000 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout:Modify: 2026-02-22 23:14:58.519170350 +0000 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout:Change: 2026-02-22 23:14:58.519170350 +0000 2026-02-22T23:19:51.855 INFO:teuthology.orchestra.run.trial040.stdout: Birth: 2026-02-22 23:14:58.519170350 +0000 2026-02-22T23:19:51.856 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T23:19:51.916 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records in 2026-02-22T23:19:51.916 INFO:teuthology.orchestra.run.trial040.stderr:1+0 records out 2026-02-22T23:19:51.916 INFO:teuthology.orchestra.run.trial040.stderr:512 bytes copied, 0.000141853 s, 3.6 MB/s 2026-02-22T23:19:51.917 DEBUG:teuthology.orchestra.run.trial040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T23:19:51.974 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:51.975 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:19:51.975 DEBUG:teuthology.orchestra.run.trial080:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T23:19:51.994 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T23:19:51.994 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_1 2026-02-22T23:19:52.052 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T23:19:52.052 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:52.052 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1305 Links: 1 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-22 23:15:17.431580114 +0000 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.053 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.054 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T23:19:52.115 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-22T23:19:52.115 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-22T23:19:52.115 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000161302 s, 3.2 MB/s 2026-02-22T23:19:52.117 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T23:19:52.174 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_2 2026-02-22T23:19:52.230 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T23:19:52.230 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:52.230 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1302 Links: 1 2026-02-22T23:19:52.231 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:52.231 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:52.231 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-22 23:15:17.431580114 +0000 2026-02-22T23:19:52.231 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.231 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.231 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.231 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T23:19:52.294 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-22T23:19:52.294 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-22T23:19:52.294 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000166131 s, 3.1 MB/s 2026-02-22T23:19:52.295 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T23:19:52.353 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_3 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-22 23:15:17.431580114 +0000 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.410 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.411 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.411 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T23:19:52.473 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-22T23:19:52.473 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-22T23:19:52.473 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000150021 s, 3.4 MB/s 2026-02-22T23:19:52.474 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T23:19:52.531 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_4 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1293 Links: 1 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-22 23:15:17.432580109 +0000 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.588 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-22 23:14:40.231763231 +0000 2026-02-22T23:19:52.589 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T23:19:52.651 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-22T23:19:52.651 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-22T23:19:52.652 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000158227 s, 3.2 MB/s 2026-02-22T23:19:52.653 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T23:19:52.712 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:52.713 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:19:52.713 DEBUG:teuthology.orchestra.run.trial189:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T23:19:52.731 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T23:19:52.731 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_1 2026-02-22T23:19:52.789 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T23:19:52.789 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:52.789 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1311 Links: 1 2026-02-22T23:19:52.789 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:52.789 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:52.790 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-22 23:15:31.824630341 +0000 2026-02-22T23:19:52.790 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-22 23:14:50.668772733 +0000 2026-02-22T23:19:52.790 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-22 23:14:50.668772733 +0000 2026-02-22T23:19:52.790 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-22 23:14:50.668772733 +0000 2026-02-22T23:19:52.790 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T23:19:52.853 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-22T23:19:52.853 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-22T23:19:52.853 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000156553 s, 3.3 MB/s 2026-02-22T23:19:52.854 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T23:19:52.911 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_2 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1315 Links: 1 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-22 23:15:31.824630341 +0000 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-22 23:14:50.668772733 +0000 2026-02-22T23:19:52.968 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-22 23:14:50.668772733 +0000 2026-02-22T23:19:52.969 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-22 23:14:50.668772733 +0000 2026-02-22T23:19:52.969 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T23:19:53.030 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-22T23:19:53.031 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-22T23:19:53.031 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000143418 s, 3.6 MB/s 2026-02-22T23:19:53.032 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T23:19:53.090 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_3 2026-02-22T23:19:53.147 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T23:19:53.147 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:53.147 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-22T23:19:53.147 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:53.147 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:53.147 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-22 23:15:31.824630341 +0000 2026-02-22T23:19:53.147 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-22 23:14:50.667772763 +0000 2026-02-22T23:19:53.148 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-22 23:14:50.667772763 +0000 2026-02-22T23:19:53.148 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-22 23:14:50.667772763 +0000 2026-02-22T23:19:53.148 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T23:19:53.211 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-22T23:19:53.212 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-22T23:19:53.212 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000145522 s, 3.5 MB/s 2026-02-22T23:19:53.213 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T23:19:53.271 DEBUG:teuthology.orchestra.run.trial189:> stat /dev/vg_nvme/lv_4 2026-02-22T23:19:53.326 INFO:teuthology.orchestra.run.trial189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout:Access: 2026-02-22 23:15:31.825630324 +0000 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout:Modify: 2026-02-22 23:14:50.667772763 +0000 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout:Change: 2026-02-22 23:14:50.667772763 +0000 2026-02-22T23:19:53.327 INFO:teuthology.orchestra.run.trial189.stdout: Birth: 2026-02-22 23:14:50.667772763 +0000 2026-02-22T23:19:53.328 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T23:19:53.389 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records in 2026-02-22T23:19:53.389 INFO:teuthology.orchestra.run.trial189.stderr:1+0 records out 2026-02-22T23:19:53.390 INFO:teuthology.orchestra.run.trial189.stderr:512 bytes copied, 0.000146464 s, 3.5 MB/s 2026-02-22T23:19:53.390 DEBUG:teuthology.orchestra.run.trial189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T23:19:53.450 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:53.450 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial040.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial080.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial189.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-22T23:19:53.450 INFO:tasks.ceph:Generating config... 2026-02-22T23:19:53.452 INFO:tasks.ceph:[client] debug monc = 20 2026-02-22T23:19:53.452 INFO:tasks.ceph:[client] debug ms = 1 2026-02-22T23:19:53.453 INFO:tasks.ceph:[client] debug objecter = 20 2026-02-22T23:19:53.453 INFO:tasks.ceph:[client] debug rados = 20 2026-02-22T23:19:53.453 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-22T23:19:53.453 INFO:tasks.ceph:[global] mon election default strategy = 1 2026-02-22T23:19:53.453 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2026-02-22T23:19:53.453 INFO:tasks.ceph:[global] ms type = async 2026-02-22T23:19:53.453 INFO:tasks.ceph:[global] osd heartbeat grace = 160 2026-02-22T23:19:53.453 INFO:tasks.ceph:[global] osd op thread timeout = 150 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2026-02-22T23:19:53.453 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-22T23:19:53.454 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-22T23:19:53.454 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-22T23:19:53.454 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-22T23:19:53.454 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2026-02-22T23:19:53.454 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] bdev async discard = True 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] bdev enable discard = True 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] debug bluefs = 1 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] debug bluestore = 1 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-22T23:19:53.454 INFO:tasks.ceph:[osd] debug rocksdb = 10 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd class default list = * 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd class load list = * 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd client watch timeout = 240 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd fast shutdown = False 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-22T23:19:53.455 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-22T23:19:53.456 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-22T23:19:53.456 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-22T23:19:53.456 INFO:tasks.ceph:Setting up mon.a... 2026-02-22T23:19:53.456 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-22T23:19:53.492 INFO:teuthology.orchestra.run.trial006.stdout:creating /etc/ceph/ceph.keyring 2026-02-22T23:19:53.495 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-22T23:19:53.571 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:19:53.636 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.6'), ('mon.b', '10.20.193.40'), ('mon.c', '10.20.193.80')] 2026-02-22T23:19:53.637 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.6,10.20.193.40,10.20.193.80', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 160, 'osd op thread timeout': 150}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd class default list': '*', 'osd class load list': '*', 'osd client watch timeout': 240, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2026-02-22T23:19:53.639 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:53.639 DEBUG:teuthology.orchestra.run.trial006:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T23:19:53.694 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.6 --add b 10.20.193.40 --add c 10.20.193.80 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:19:53.769 INFO:teuthology.orchestra.run.trial006.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:monmaptool: generated fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:setting min_mon_release = quincy 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:epoch 0 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:min_mon_release 17 (quincy) 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:election_strategy: 1 2026-02-22T23:19:53.770 INFO:teuthology.orchestra.run.trial006.stdout:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:19:53.771 INFO:teuthology.orchestra.run.trial006.stdout:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:19:53.771 INFO:teuthology.orchestra.run.trial006.stdout:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:19:53.771 INFO:teuthology.orchestra.run.trial006.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-22T23:19:53.772 DEBUG:teuthology.orchestra.run.trial006:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T23:19:53.830 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f24f9547-07c5-4c58-99d4-b3bf9b186dd1... 2026-02-22T23:19:53.831 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T23:19:53.871 DEBUG:teuthology.orchestra.run.trial040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T23:19:53.873 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T23:19:53.875 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout:[global] 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout: chdir = "" 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout: auth supported = cephx 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout: filestore xattr use omap = true 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.910 INFO:teuthology.orchestra.run.trial006.stdout: mon clock drift allowed = 1.000 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: osd crush chooseleaf type = 0 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: auth debug = true 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: ms die on old message = true 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: ms die on bug = true 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: mon pg warn max object skew = 0 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.911 INFO:teuthology.orchestra.run.trial006.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: osd pool default size = 2 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: mon osd allow primary affinity = true 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: mon osd allow pg remap = true 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on legacy crush tunables = false 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on crush straw calc version zero = false 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on no sortbitwise = false 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on osd down out interval zero = false 2026-02-22T23:19:53.912 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on too few osds = false 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: mon_allow_pool_size_one = true 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: osd default data pool replay window = 5 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: mon allow pool delete = true 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: mon cluster log file level = debug 2026-02-22T23:19:53.913 INFO:teuthology.orchestra.run.trial006.stdout: debug asserts on shutdown = true 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: mon health detail to clog = false 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: mon host = "10.20.193.6,10.20.193.40,10.20.193.80" 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: mon client directed command retry = 5 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: mon election default strategy = 1 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: ms inject socket failures = 5000 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: ms type = async 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: osd heartbeat grace = 160 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: osd op thread timeout = 150 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: fsid = f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout:[osd] 2026-02-22T23:19:53.914 INFO:teuthology.orchestra.run.trial006.stdout: osd journal size = 100 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd scrub load threshold = 5.0 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd scrub max interval = 600 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd mclock profile = high_recovery_ops 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd mclock skip benchmark = True 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd recover clone overlap = true 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd recovery max chunk = 1048576 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd debug shutdown = true 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd debug op order = true 2026-02-22T23:19:53.915 INFO:teuthology.orchestra.run.trial006.stdout: osd debug verify stray on activate = true 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: osd debug trim objects = true 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: osd open classes on start = true 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: osd debug pg log writeout = true 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: osd deep scrub update digest min age = 30 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: osd map max advance = 10 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: journal zero on create = true 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: filestore ondisk finisher threads = 3 2026-02-22T23:19:53.916 INFO:teuthology.orchestra.run.trial006.stdout: filestore apply finisher threads = 3 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: bdev debug aio = true 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: osd debug misdirected ops = true 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: bdev async discard = True 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: bdev enable discard = True 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: bluestore block size = 96636764160 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: bluestore fsck on mount = True 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: bluestore zero block detection = True 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: debug bluefs = 1 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: debug bluestore = 1 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: debug monc = 20 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: debug ms = 1 2026-02-22T23:19:53.917 INFO:teuthology.orchestra.run.trial006.stdout: debug osd = 20 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: debug rocksdb = 10 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: mon osd full ratio = 0.9 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: mon osd nearfull ratio = 0.8 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd class default list = * 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd class load list = * 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd client watch timeout = 240 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd debug verify cached snaps = True 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd debug verify missing on start = True 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd failsafe full ratio = 0.95 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd fast shutdown = False 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd mclock override recovery settings = True 2026-02-22T23:19:53.918 INFO:teuthology.orchestra.run.trial006.stdout: osd memory target = 1610612736 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: osd objectstore = bluestore 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: osd op queue = debug_random 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: osd op queue cut off = debug_random 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: osd shutdown pgref assert = True 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout:[mgr] 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: debug ms = 1 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: debug mgr = 20 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: debug mon = 20 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: debug auth = 20 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.919 INFO:teuthology.orchestra.run.trial006.stdout: mgr/telemetry/nag = false 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout:[mon] 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: debug ms = 1 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: debug mon = 20 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: debug paxos = 20 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: debug auth = 20 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: mon data avail warn = 5 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: mon mgr mkfs grace = 240 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: mon osd reporter subtree level = osd 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: mon osd prime pg temp = true 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.920 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: # 1m isn't quite enough 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: mon_down_mkfs_grace = 2m 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: mon_warn_on_filestore_osds = false 2026-02-22T23:19:53.921 INFO:teuthology.orchestra.run.trial006.stdout: mon min osdmap epochs = 50 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: mon osd crush smoke test = False 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: mon osdmap full prune interval = 2 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: mon osdmap full prune min = 15 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: mon osdmap full prune txsize = 2 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: mon scrub interval = 300 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: mon warn on pool no app = False 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: paxos service trim min = 10 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout:[client] 2026-02-22T23:19:53.922 INFO:teuthology.orchestra.run.trial006.stdout: rgw cache enabled = true 2026-02-22T23:19:53.923 INFO:teuthology.orchestra.run.trial006.stdout: rgw enable ops log = true 2026-02-22T23:19:53.923 INFO:teuthology.orchestra.run.trial006.stdout: rgw enable usage log = true 2026-02-22T23:19:53.923 INFO:teuthology.orchestra.run.trial006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T23:19:53.923 INFO:teuthology.orchestra.run.trial006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T23:19:53.923 INFO:teuthology.orchestra.run.trial006.stdout: debug monc = 20 2026-02-22T23:19:53.923 INFO:teuthology.orchestra.run.trial006.stdout: debug ms = 1 2026-02-22T23:19:53.923 INFO:teuthology.orchestra.run.trial006.stdout: debug objecter = 20 2026-02-22T23:19:53.924 INFO:teuthology.orchestra.run.trial006.stdout: debug rados = 20 2026-02-22T23:19:53.924 INFO:teuthology.orchestra.run.trial006.stdout:[mon.a] 2026-02-22T23:19:53.924 INFO:teuthology.orchestra.run.trial006.stdout:[mon.b] 2026-02-22T23:19:53.924 INFO:teuthology.orchestra.run.trial006.stdout:[mon.c] 2026-02-22T23:19:53.925 INFO:teuthology.orchestra.run.trial189.stdout:[global] 2026-02-22T23:19:53.925 INFO:teuthology.orchestra.run.trial189.stdout: chdir = "" 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: auth supported = cephx 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: filestore xattr use omap = true 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: mon clock drift allowed = 1.000 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: osd crush chooseleaf type = 0 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: auth debug = true 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.926 INFO:teuthology.orchestra.run.trial189.stdout: ms die on old message = true 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: ms die on bug = true 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: mon pg warn max object skew = 0 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: osd pool default size = 2 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: mon osd allow primary affinity = true 2026-02-22T23:19:53.927 INFO:teuthology.orchestra.run.trial189.stdout: mon osd allow pg remap = true 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on legacy crush tunables = false 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on crush straw calc version zero = false 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on no sortbitwise = false 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on osd down out interval zero = false 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on too few osds = false 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: mon_allow_pool_size_one = true 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: osd default data pool replay window = 5 2026-02-22T23:19:53.928 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: mon allow pool delete = true 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: mon cluster log file level = debug 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: debug asserts on shutdown = true 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: mon health detail to clog = false 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: mon host = "10.20.193.6,10.20.193.40,10.20.193.80" 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: mon client directed command retry = 5 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: mon election default strategy = 1 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: ms inject socket failures = 5000 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: ms type = async 2026-02-22T23:19:53.929 INFO:teuthology.orchestra.run.trial189.stdout: osd heartbeat grace = 160 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: osd op thread timeout = 150 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: fsid = f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout:[osd] 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: osd journal size = 100 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: osd scrub load threshold = 5.0 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: osd scrub max interval = 600 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: osd mclock profile = high_recovery_ops 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: osd mclock skip benchmark = True 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.930 INFO:teuthology.orchestra.run.trial189.stdout: osd recover clone overlap = true 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: osd recovery max chunk = 1048576 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: osd debug shutdown = true 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: osd debug op order = true 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: osd debug verify stray on activate = true 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: osd debug trim objects = true 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.931 INFO:teuthology.orchestra.run.trial189.stdout: osd open classes on start = true 2026-02-22T23:19:53.932 INFO:teuthology.orchestra.run.trial189.stdout: osd debug pg log writeout = true 2026-02-22T23:19:53.932 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.932 INFO:teuthology.orchestra.run.trial189.stdout: osd deep scrub update digest min age = 30 2026-02-22T23:19:53.932 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.932 INFO:teuthology.orchestra.run.trial189.stdout: osd map max advance = 10 2026-02-22T23:19:53.932 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.932 INFO:teuthology.orchestra.run.trial189.stdout: journal zero on create = true 2026-02-22T23:19:53.933 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.933 INFO:teuthology.orchestra.run.trial189.stdout: filestore ondisk finisher threads = 3 2026-02-22T23:19:53.933 INFO:teuthology.orchestra.run.trial189.stdout: filestore apply finisher threads = 3 2026-02-22T23:19:53.933 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.933 INFO:teuthology.orchestra.run.trial189.stdout: bdev debug aio = true 2026-02-22T23:19:53.933 INFO:teuthology.orchestra.run.trial189.stdout: osd debug misdirected ops = true 2026-02-22T23:19:53.934 INFO:teuthology.orchestra.run.trial189.stdout: bdev async discard = True 2026-02-22T23:19:53.934 INFO:teuthology.orchestra.run.trial189.stdout: bdev enable discard = True 2026-02-22T23:19:53.934 INFO:teuthology.orchestra.run.trial189.stdout: bluestore block size = 96636764160 2026-02-22T23:19:53.934 INFO:teuthology.orchestra.run.trial189.stdout: bluestore fsck on mount = True 2026-02-22T23:19:53.934 INFO:teuthology.orchestra.run.trial189.stdout: bluestore zero block detection = True 2026-02-22T23:19:53.934 INFO:teuthology.orchestra.run.trial189.stdout: debug bluefs = 1 2026-02-22T23:19:53.934 INFO:teuthology.orchestra.run.trial189.stdout: debug bluestore = 1 2026-02-22T23:19:53.935 INFO:teuthology.orchestra.run.trial189.stdout: debug monc = 20 2026-02-22T23:19:53.935 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-22T23:19:53.935 INFO:teuthology.orchestra.run.trial189.stdout: debug osd = 20 2026-02-22T23:19:53.935 INFO:teuthology.orchestra.run.trial189.stdout: debug rocksdb = 10 2026-02-22T23:19:53.935 INFO:teuthology.orchestra.run.trial189.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T23:19:53.935 INFO:teuthology.orchestra.run.trial189.stdout: mon osd full ratio = 0.9 2026-02-22T23:19:53.936 INFO:teuthology.orchestra.run.trial189.stdout: mon osd nearfull ratio = 0.8 2026-02-22T23:19:53.936 INFO:teuthology.orchestra.run.trial189.stdout: osd class default list = * 2026-02-22T23:19:53.936 INFO:teuthology.orchestra.run.trial189.stdout: osd class load list = * 2026-02-22T23:19:53.936 INFO:teuthology.orchestra.run.trial189.stdout: osd client watch timeout = 240 2026-02-22T23:19:53.936 INFO:teuthology.orchestra.run.trial189.stdout: osd debug verify cached snaps = True 2026-02-22T23:19:53.936 INFO:teuthology.orchestra.run.trial189.stdout: osd debug verify missing on start = True 2026-02-22T23:19:53.936 INFO:teuthology.orchestra.run.trial189.stdout: osd failsafe full ratio = 0.95 2026-02-22T23:19:53.937 INFO:teuthology.orchestra.run.trial189.stdout: osd fast shutdown = False 2026-02-22T23:19:53.937 INFO:teuthology.orchestra.run.trial189.stdout: osd mclock override recovery settings = True 2026-02-22T23:19:53.937 INFO:teuthology.orchestra.run.trial189.stdout: osd memory target = 1610612736 2026-02-22T23:19:53.937 INFO:teuthology.orchestra.run.trial189.stdout: osd objectstore = bluestore 2026-02-22T23:19:53.937 INFO:teuthology.orchestra.run.trial189.stdout: osd op queue = debug_random 2026-02-22T23:19:53.937 INFO:teuthology.orchestra.run.trial189.stdout: osd op queue cut off = debug_random 2026-02-22T23:19:53.937 INFO:teuthology.orchestra.run.trial189.stdout: osd shutdown pgref assert = True 2026-02-22T23:19:53.938 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.938 INFO:teuthology.orchestra.run.trial189.stdout:[mgr] 2026-02-22T23:19:53.938 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-22T23:19:53.938 INFO:teuthology.orchestra.run.trial189.stdout: debug mgr = 20 2026-02-22T23:19:53.938 INFO:teuthology.orchestra.run.trial189.stdout: debug mon = 20 2026-02-22T23:19:53.938 INFO:teuthology.orchestra.run.trial189.stdout: debug auth = 20 2026-02-22T23:19:53.939 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.939 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.939 INFO:teuthology.orchestra.run.trial189.stdout: mgr/telemetry/nag = false 2026-02-22T23:19:53.939 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.939 INFO:teuthology.orchestra.run.trial189.stdout:[mon] 2026-02-22T23:19:53.939 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-22T23:19:53.939 INFO:teuthology.orchestra.run.trial189.stdout: debug mon = 20 2026-02-22T23:19:53.940 INFO:teuthology.orchestra.run.trial189.stdout: debug paxos = 20 2026-02-22T23:19:53.940 INFO:teuthology.orchestra.run.trial189.stdout: debug auth = 20 2026-02-22T23:19:53.940 INFO:teuthology.orchestra.run.trial189.stdout: mon data avail warn = 5 2026-02-22T23:19:53.940 INFO:teuthology.orchestra.run.trial189.stdout: mon mgr mkfs grace = 240 2026-02-22T23:19:53.940 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.940 INFO:teuthology.orchestra.run.trial189.stdout: mon osd reporter subtree level = osd 2026-02-22T23:19:53.941 INFO:teuthology.orchestra.run.trial189.stdout: mon osd prime pg temp = true 2026-02-22T23:19:53.941 INFO:teuthology.orchestra.run.trial189.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.941 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.941 INFO:teuthology.orchestra.run.trial189.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T23:19:53.941 INFO:teuthology.orchestra.run.trial189.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T23:19:53.941 INFO:teuthology.orchestra.run.trial189.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T23:19:53.942 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.942 INFO:teuthology.orchestra.run.trial189.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T23:19:53.942 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T23:19:53.942 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T23:19:53.942 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.942 INFO:teuthology.orchestra.run.trial189.stdout: # 1m isn't quite enough 2026-02-22T23:19:53.943 INFO:teuthology.orchestra.run.trial189.stdout: mon_down_mkfs_grace = 2m 2026-02-22T23:19:53.943 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.943 INFO:teuthology.orchestra.run.trial189.stdout: mon_warn_on_filestore_osds = false 2026-02-22T23:19:53.943 INFO:teuthology.orchestra.run.trial189.stdout: mon min osdmap epochs = 50 2026-02-22T23:19:53.943 INFO:teuthology.orchestra.run.trial189.stdout: mon osd crush smoke test = False 2026-02-22T23:19:53.943 INFO:teuthology.orchestra.run.trial189.stdout: mon osdmap full prune interval = 2 2026-02-22T23:19:53.943 INFO:teuthology.orchestra.run.trial189.stdout: mon osdmap full prune min = 15 2026-02-22T23:19:53.944 INFO:teuthology.orchestra.run.trial189.stdout: mon osdmap full prune txsize = 2 2026-02-22T23:19:53.944 INFO:teuthology.orchestra.run.trial189.stdout: mon scrub interval = 300 2026-02-22T23:19:53.944 INFO:teuthology.orchestra.run.trial189.stdout: mon warn on pool no app = False 2026-02-22T23:19:53.944 INFO:teuthology.orchestra.run.trial189.stdout: paxos service trim min = 10 2026-02-22T23:19:53.944 INFO:teuthology.orchestra.run.trial189.stdout: 2026-02-22T23:19:53.944 INFO:teuthology.orchestra.run.trial189.stdout:[client] 2026-02-22T23:19:53.944 INFO:teuthology.orchestra.run.trial189.stdout: rgw cache enabled = true 2026-02-22T23:19:53.945 INFO:teuthology.orchestra.run.trial189.stdout: rgw enable ops log = true 2026-02-22T23:19:53.945 INFO:teuthology.orchestra.run.trial189.stdout: rgw enable usage log = true 2026-02-22T23:19:53.945 INFO:teuthology.orchestra.run.trial189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T23:19:53.945 INFO:teuthology.orchestra.run.trial189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T23:19:53.945 INFO:teuthology.orchestra.run.trial189.stdout: debug monc = 20 2026-02-22T23:19:53.945 INFO:teuthology.orchestra.run.trial189.stdout: debug ms = 1 2026-02-22T23:19:53.946 INFO:teuthology.orchestra.run.trial189.stdout: debug objecter = 20 2026-02-22T23:19:53.946 INFO:teuthology.orchestra.run.trial189.stdout: debug rados = 20 2026-02-22T23:19:53.946 INFO:teuthology.orchestra.run.trial189.stdout:[mon.a] 2026-02-22T23:19:53.946 INFO:teuthology.orchestra.run.trial189.stdout:[mon.b] 2026-02-22T23:19:53.946 INFO:teuthology.orchestra.run.trial189.stdout:[mon.c] 2026-02-22T23:19:53.946 INFO:teuthology.orchestra.run.trial080.stdout:[global] 2026-02-22T23:19:53.947 INFO:teuthology.orchestra.run.trial080.stdout: chdir = "" 2026-02-22T23:19:53.947 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T23:19:53.947 INFO:teuthology.orchestra.run.trial080.stdout: auth supported = cephx 2026-02-22T23:19:53.947 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.947 INFO:teuthology.orchestra.run.trial080.stdout: filestore xattr use omap = true 2026-02-22T23:19:53.947 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.947 INFO:teuthology.orchestra.run.trial080.stdout: mon clock drift allowed = 1.000 2026-02-22T23:19:53.948 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.948 INFO:teuthology.orchestra.run.trial080.stdout: osd crush chooseleaf type = 0 2026-02-22T23:19:53.948 INFO:teuthology.orchestra.run.trial080.stdout: auth debug = true 2026-02-22T23:19:53.948 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.948 INFO:teuthology.orchestra.run.trial080.stdout: ms die on old message = true 2026-02-22T23:19:53.948 INFO:teuthology.orchestra.run.trial080.stdout: ms die on bug = true 2026-02-22T23:19:53.948 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: mon pg warn max object skew = 0 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: osd pool default size = 2 2026-02-22T23:19:53.949 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.950 INFO:teuthology.orchestra.run.trial080.stdout: mon osd allow primary affinity = true 2026-02-22T23:19:53.950 INFO:teuthology.orchestra.run.trial080.stdout: mon osd allow pg remap = true 2026-02-22T23:19:53.950 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on legacy crush tunables = false 2026-02-22T23:19:53.950 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on crush straw calc version zero = false 2026-02-22T23:19:53.950 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on no sortbitwise = false 2026-02-22T23:19:53.950 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on osd down out interval zero = false 2026-02-22T23:19:53.950 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on too few osds = false 2026-02-22T23:19:53.951 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T23:19:53.951 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T23:19:53.951 INFO:teuthology.orchestra.run.trial080.stdout: mon_allow_pool_size_one = true 2026-02-22T23:19:53.951 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.951 INFO:teuthology.orchestra.run.trial080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T23:19:53.951 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.951 INFO:teuthology.orchestra.run.trial080.stdout: osd default data pool replay window = 5 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: mon allow pool delete = true 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: mon cluster log file level = debug 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: debug asserts on shutdown = true 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: mon health detail to clog = false 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: mon host = "10.20.193.6,10.20.193.40,10.20.193.80" 2026-02-22T23:19:53.952 INFO:teuthology.orchestra.run.trial080.stdout: mon client directed command retry = 5 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: mon election default strategy = 1 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: ms inject socket failures = 5000 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: ms type = async 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: osd heartbeat grace = 160 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: osd op thread timeout = 150 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: fsid = f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout:[osd] 2026-02-22T23:19:53.953 INFO:teuthology.orchestra.run.trial080.stdout: osd journal size = 100 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: osd scrub load threshold = 5.0 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: osd scrub max interval = 600 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: osd mclock profile = high_recovery_ops 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: osd mclock skip benchmark = True 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: osd recover clone overlap = true 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: osd recovery max chunk = 1048576 2026-02-22T23:19:53.954 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd debug shutdown = true 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd debug op order = true 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd debug verify stray on activate = true 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd debug trim objects = true 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd open classes on start = true 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd debug pg log writeout = true 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd deep scrub update digest min age = 30 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.955 INFO:teuthology.orchestra.run.trial080.stdout: osd map max advance = 10 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: journal zero on create = true 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: filestore ondisk finisher threads = 3 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: filestore apply finisher threads = 3 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: bdev debug aio = true 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: osd debug misdirected ops = true 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: bdev async discard = True 2026-02-22T23:19:53.956 INFO:teuthology.orchestra.run.trial080.stdout: bdev enable discard = True 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: bluestore block size = 96636764160 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: bluestore fsck on mount = True 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: bluestore zero block detection = True 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: debug bluefs = 1 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: debug bluestore = 1 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: debug monc = 20 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: debug osd = 20 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: debug rocksdb = 10 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: mon osd full ratio = 0.9 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: mon osd nearfull ratio = 0.8 2026-02-22T23:19:53.957 INFO:teuthology.orchestra.run.trial080.stdout: osd class default list = * 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd class load list = * 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd client watch timeout = 240 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd debug verify cached snaps = True 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd debug verify missing on start = True 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd failsafe full ratio = 0.95 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd fast shutdown = False 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd mclock override recovery settings = True 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd memory target = 1610612736 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd objectstore = bluestore 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd op queue = debug_random 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd op queue cut off = debug_random 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: osd shutdown pgref assert = True 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout:[mgr] 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: debug mgr = 20 2026-02-22T23:19:53.958 INFO:teuthology.orchestra.run.trial080.stdout: debug mon = 20 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: debug auth = 20 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mgr/telemetry/nag = false 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout:[mon] 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: debug mon = 20 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: debug paxos = 20 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: debug auth = 20 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon data avail warn = 5 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon mgr mkfs grace = 240 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon osd reporter subtree level = osd 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon osd prime pg temp = true 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T23:19:53.959 INFO:teuthology.orchestra.run.trial080.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: # 1m isn't quite enough 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon_down_mkfs_grace = 2m 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_filestore_osds = false 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon min osdmap epochs = 50 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon osd crush smoke test = False 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon osdmap full prune interval = 2 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon osdmap full prune min = 15 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon osdmap full prune txsize = 2 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon scrub interval = 300 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on pool no app = False 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: paxos service trim min = 10 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-22T23:19:53.960 INFO:teuthology.orchestra.run.trial080.stdout:[client] 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: rgw cache enabled = true 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: rgw enable ops log = true 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: rgw enable usage log = true 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: debug monc = 20 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: debug objecter = 20 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout: debug rados = 20 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout:[mon.a] 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout:[mon.b] 2026-02-22T23:19:53.961 INFO:teuthology.orchestra.run.trial080.stdout:[mon.c] 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout:[global] 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: chdir = "" 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: auth supported = cephx 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: filestore xattr use omap = true 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: mon clock drift allowed = 1.000 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: osd crush chooseleaf type = 0 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: auth debug = true 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: ms die on old message = true 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: ms die on bug = true 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T23:19:53.963 INFO:teuthology.orchestra.run.trial040.stdout: mon pg warn max object skew = 0 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: osd pool default size = 2 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon osd allow primary affinity = true 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon osd allow pg remap = true 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon warn on legacy crush tunables = false 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon warn on crush straw calc version zero = false 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon warn on no sortbitwise = false 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon warn on osd down out interval zero = false 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon warn on too few osds = false 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: mon_allow_pool_size_one = true 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.964 INFO:teuthology.orchestra.run.trial040.stdout: osd default data pool replay window = 5 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: mon allow pool delete = true 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: mon cluster log file level = debug 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: debug asserts on shutdown = true 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: mon health detail to clog = false 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: mon host = "10.20.193.6,10.20.193.40,10.20.193.80" 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: mon client directed command retry = 5 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: mon election default strategy = 1 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: ms inject socket failures = 5000 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: ms type = async 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: osd heartbeat grace = 160 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: osd op thread timeout = 150 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: fsid = f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout:[osd] 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: osd journal size = 100 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: osd scrub load threshold = 5.0 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: osd scrub max interval = 600 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: osd mclock profile = high_recovery_ops 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: osd mclock skip benchmark = True 2026-02-22T23:19:53.965 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd recover clone overlap = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd recovery max chunk = 1048576 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd debug shutdown = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd debug op order = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd debug verify stray on activate = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd debug trim objects = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd open classes on start = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd debug pg log writeout = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd deep scrub update digest min age = 30 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd map max advance = 10 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: journal zero on create = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: filestore ondisk finisher threads = 3 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: filestore apply finisher threads = 3 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: bdev debug aio = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: osd debug misdirected ops = true 2026-02-22T23:19:53.966 INFO:teuthology.orchestra.run.trial040.stdout: bdev async discard = True 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: bdev enable discard = True 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: bluestore block size = 96636764160 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: bluestore fsck on mount = True 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: bluestore zero block detection = True 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: debug bluefs = 1 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: debug bluestore = 1 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: debug monc = 20 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: debug ms = 1 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: debug osd = 20 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: debug rocksdb = 10 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: mon osd full ratio = 0.9 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: mon osd nearfull ratio = 0.8 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd class default list = * 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd class load list = * 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd client watch timeout = 240 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd debug verify cached snaps = True 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd debug verify missing on start = True 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd failsafe full ratio = 0.95 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd fast shutdown = False 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd mclock override recovery settings = True 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd memory target = 1610612736 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd objectstore = bluestore 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd op queue = debug_random 2026-02-22T23:19:53.967 INFO:teuthology.orchestra.run.trial040.stdout: osd op queue cut off = debug_random 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: osd shutdown pgref assert = True 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout:[mgr] 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug ms = 1 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug mgr = 20 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug mon = 20 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug auth = 20 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mgr/telemetry/nag = false 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout:[mon] 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug ms = 1 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug mon = 20 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug paxos = 20 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: debug auth = 20 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon data avail warn = 5 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon mgr mkfs grace = 240 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon osd reporter subtree level = osd 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon osd prime pg temp = true 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.968 INFO:teuthology.orchestra.run.trial040.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: # 1m isn't quite enough 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon_down_mkfs_grace = 2m 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon_warn_on_filestore_osds = false 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon min osdmap epochs = 50 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon osd crush smoke test = False 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon osdmap full prune interval = 2 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon osdmap full prune min = 15 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon osdmap full prune txsize = 2 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon scrub interval = 300 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: mon warn on pool no app = False 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: paxos service trim min = 10 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout:[client] 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: rgw cache enabled = true 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: rgw enable ops log = true 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: rgw enable usage log = true 2026-02-22T23:19:53.969 INFO:teuthology.orchestra.run.trial040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout: debug monc = 20 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout: debug ms = 1 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout: debug objecter = 20 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout: debug rados = 20 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout:[mon.a] 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout:[mon.b] 2026-02-22T23:19:53.970 INFO:teuthology.orchestra.run.trial040.stdout:[mon.c] 2026-02-22T23:19:53.971 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-22T23:19:53.971 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-22T23:19:54.045 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-22T23:19:54.045 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:54.046 DEBUG:teuthology.orchestra.run.trial006:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-22T23:19:54.101 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:54.102 DEBUG:teuthology.orchestra.run.trial006:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-22T23:19:54.157 INFO:tasks.ceph:Sending monmap to node ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:19:54.158 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:54.158 DEBUG:teuthology.orchestra.run.trial006:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T23:19:54.158 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:19:54.230 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:19:54.230 DEBUG:teuthology.orchestra.run.trial006:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:19:54.286 INFO:tasks.ceph:Sending monmap to node ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:19:54.286 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:19:54.286 DEBUG:teuthology.orchestra.run.trial040:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T23:19:54.286 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:19:54.321 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:19:54.321 DEBUG:teuthology.orchestra.run.trial040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:19:54.377 INFO:tasks.ceph:Sending monmap to node ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:19:54.378 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:19:54.378 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T23:19:54.378 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:19:54.413 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:19:54.413 DEBUG:teuthology.orchestra.run.trial080:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:19:54.470 INFO:tasks.ceph:Sending monmap to node ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:19:54.471 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:19:54.471 DEBUG:teuthology.orchestra.run.trial189:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T23:19:54.471 DEBUG:teuthology.orchestra.run.trial189:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:19:54.504 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:19:54.504 DEBUG:teuthology.orchestra.run.trial189:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:19:54.561 INFO:tasks.ceph:Setting up mon nodes... 2026-02-22T23:19:54.561 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-22T23:19:54.561 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2026-02-22T23:19:54.606 INFO:teuthology.orchestra.run.trial006.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2026-02-22T23:19:54.608 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T23:19:54.651 INFO:teuthology.orchestra.run.trial189.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T23:19:54.653 INFO:tasks.ceph:Setting up mds nodes... 2026-02-22T23:19:54.653 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-22T23:19:54.653 DEBUG:teuthology.orchestra.run.trial189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-22T23:19:54.729 INFO:teuthology.orchestra.run.trial189.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-22T23:19:54.740 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-22T23:19:54.741 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial040.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial080.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@trial189.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2026-02-22T23:19:54.741 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-22T23:19:54.766 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:54.767 INFO:tasks.ceph:role: osd.0 2026-02-22T23:19:54.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:19:54.767 DEBUG:teuthology.orchestra.run.trial006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T23:19:54.831 INFO:teuthology.orchestra.run.trial006.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:19:54.831 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:19:54.831 INFO:teuthology.orchestra.run.trial006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:19:54.831 INFO:teuthology.orchestra.run.trial006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:19:54.831 INFO:teuthology.orchestra.run.trial006.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:19:54.831 INFO:teuthology.orchestra.run.trial006.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:19:54.831 INFO:teuthology.orchestra.run.trial006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:19:54.832 INFO:teuthology.orchestra.run.trial006.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:19:54.832 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:19:54.832 INFO:teuthology.orchestra.run.trial006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:19:55.454 INFO:teuthology.orchestra.run.trial006.stdout:Discarding blocks...Done. 2026-02-22T23:19:55.456 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial006.front.sepia.ceph.com -o noatime 2026-02-22T23:19:55.457 DEBUG:teuthology.orchestra.run.trial006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-22T23:19:55.525 DEBUG:teuthology.orchestra.run.trial006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-22T23:19:55.592 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-22T23:19:55.656 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:55.656 INFO:tasks.ceph:role: osd.4 2026-02-22T23:19:55.657 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:19:55.657 DEBUG:teuthology.orchestra.run.trial006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T23:19:55.721 INFO:teuthology.orchestra.run.trial006.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:19:55.721 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:19:55.721 INFO:teuthology.orchestra.run.trial006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:19:55.721 INFO:teuthology.orchestra.run.trial006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:19:55.721 INFO:teuthology.orchestra.run.trial006.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:19:55.722 INFO:teuthology.orchestra.run.trial006.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:19:55.722 INFO:teuthology.orchestra.run.trial006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:19:55.722 INFO:teuthology.orchestra.run.trial006.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:19:55.722 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:19:55.722 INFO:teuthology.orchestra.run.trial006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:19:56.281 INFO:teuthology.orchestra.run.trial006.stdout:Discarding blocks...Done. 2026-02-22T23:19:56.283 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial006.front.sepia.ceph.com -o noatime 2026-02-22T23:19:56.283 DEBUG:teuthology.orchestra.run.trial006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2026-02-22T23:19:56.351 DEBUG:teuthology.orchestra.run.trial006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2026-02-22T23:19:56.417 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2026-02-22T23:19:56.482 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:56.482 INFO:tasks.ceph:role: osd.8 2026-02-22T23:19:56.482 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:19:56.482 DEBUG:teuthology.orchestra.run.trial006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:19:56.548 INFO:teuthology.orchestra.run.trial006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:19:57.160 INFO:teuthology.orchestra.run.trial006.stdout:Discarding blocks...Done. 2026-02-22T23:19:57.162 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial006.front.sepia.ceph.com -o noatime 2026-02-22T23:19:57.162 DEBUG:teuthology.orchestra.run.trial006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2026-02-22T23:19:57.229 DEBUG:teuthology.orchestra.run.trial006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2026-02-22T23:19:57.295 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2026-02-22T23:19:57.360 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2026-02-22T23:19:57.360 INFO:tasks.ceph:role: osd.12 2026-02-22T23:19:57.360 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:19:57.360 DEBUG:teuthology.orchestra.run.trial006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:19:57.425 INFO:teuthology.orchestra.run.trial006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:19:57.982 INFO:teuthology.orchestra.run.trial006.stdout:Discarding blocks...Done. 2026-02-22T23:19:57.984 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial006.front.sepia.ceph.com -o noatime 2026-02-22T23:19:57.984 DEBUG:teuthology.orchestra.run.trial006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2026-02-22T23:19:58.053 DEBUG:teuthology.orchestra.run.trial006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2026-02-22T23:19:58.119 DEBUG:teuthology.orchestra.run.trial006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:19:58.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:19:58.196+0000 7f9566810780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-22T23:19:58.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:19:58.196+0000 7f9566810780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-22T23:19:58.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:19:58.196+0000 7f9566810780 -1 bdev(0x55cf49f73800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-22T23:19:58.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:19:58.196+0000 7f9566810780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-22T23:20:02.080 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-22T23:20:02.146 DEBUG:teuthology.orchestra.run.trial006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:02.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:02.223+0000 7f5d9ae15780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-22T23:20:02.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:02.223+0000 7f5d9ae15780 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-22T23:20:02.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:02.223+0000 7f5d9ae15780 -1 bdev(0x55ac077df800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:02.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:02.223+0000 7f5d9ae15780 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-22T23:20:06.104 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-22T23:20:06.169 DEBUG:teuthology.orchestra.run.trial006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:06.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:06.244+0000 7fceb74fe780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2026-02-22T23:20:06.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:06.245+0000 7fceb74fe780 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2026-02-22T23:20:06.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:06.245+0000 7fceb74fe780 -1 bdev(0x55e675297800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:06.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:06.245+0000 7fceb74fe780 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2026-02-22T23:20:10.134 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2026-02-22T23:20:10.200 DEBUG:teuthology.orchestra.run.trial006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:10.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:10.276+0000 7f8909e0d780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2026-02-22T23:20:10.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:10.276+0000 7f8909e0d780 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2026-02-22T23:20:10.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:10.276+0000 7f8909e0d780 -1 bdev(0x55c4214b1800 /var/lib/ceph/osd/ceph-12/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:10.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:20:10.276+0000 7f8909e0d780 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2026-02-22T23:20:14.172 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2026-02-22T23:20:14.237 DEBUG:teuthology.orchestra.run.trial040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:14.262 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:14.263 INFO:tasks.ceph:role: osd.1 2026-02-22T23:20:14.263 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:20:14.263 DEBUG:teuthology.orchestra.run.trial040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:14.327 INFO:teuthology.orchestra.run.trial040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:14.901 INFO:teuthology.orchestra.run.trial040.stdout:Discarding blocks...Done. 2026-02-22T23:20:14.903 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial040.front.sepia.ceph.com -o noatime 2026-02-22T23:20:14.903 DEBUG:teuthology.orchestra.run.trial040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:14.970 DEBUG:teuthology.orchestra.run.trial040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:15.036 DEBUG:teuthology.orchestra.run.trial040:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-22T23:20:15.100 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:15.100 INFO:tasks.ceph:role: osd.5 2026-02-22T23:20:15.100 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:20:15.100 DEBUG:teuthology.orchestra.run.trial040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T23:20:15.167 INFO:teuthology.orchestra.run.trial040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:15.167 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:15.167 INFO:teuthology.orchestra.run.trial040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:15.167 INFO:teuthology.orchestra.run.trial040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:15.168 INFO:teuthology.orchestra.run.trial040.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:15.168 INFO:teuthology.orchestra.run.trial040.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:15.168 INFO:teuthology.orchestra.run.trial040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:15.168 INFO:teuthology.orchestra.run.trial040.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:15.168 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:15.168 INFO:teuthology.orchestra.run.trial040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:15.733 INFO:teuthology.orchestra.run.trial040.stdout:Discarding blocks...Done. 2026-02-22T23:20:15.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial040.front.sepia.ceph.com -o noatime 2026-02-22T23:20:15.735 DEBUG:teuthology.orchestra.run.trial040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2026-02-22T23:20:15.803 DEBUG:teuthology.orchestra.run.trial040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2026-02-22T23:20:15.870 DEBUG:teuthology.orchestra.run.trial040:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2026-02-22T23:20:15.934 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:15.935 INFO:tasks.ceph:role: osd.9 2026-02-22T23:20:15.935 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:20:15.935 DEBUG:teuthology.orchestra.run.trial040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-22T23:20:15.998 INFO:teuthology.orchestra.run.trial040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:15.999 INFO:teuthology.orchestra.run.trial040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:16.607 INFO:teuthology.orchestra.run.trial040.stdout:Discarding blocks...Done. 2026-02-22T23:20:16.609 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial040.front.sepia.ceph.com -o noatime 2026-02-22T23:20:16.609 DEBUG:teuthology.orchestra.run.trial040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2026-02-22T23:20:16.677 DEBUG:teuthology.orchestra.run.trial040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2026-02-22T23:20:16.744 DEBUG:teuthology.orchestra.run.trial040:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2026-02-22T23:20:16.808 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:16.808 INFO:tasks.ceph:role: osd.13 2026-02-22T23:20:16.809 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:20:16.809 DEBUG:teuthology.orchestra.run.trial040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:16.873 INFO:teuthology.orchestra.run.trial040.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:16.874 INFO:teuthology.orchestra.run.trial040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:17.431 INFO:teuthology.orchestra.run.trial040.stdout:Discarding blocks...Done. 2026-02-22T23:20:17.432 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial040.front.sepia.ceph.com -o noatime 2026-02-22T23:20:17.432 DEBUG:teuthology.orchestra.run.trial040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2026-02-22T23:20:17.500 DEBUG:teuthology.orchestra.run.trial040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2026-02-22T23:20:17.567 DEBUG:teuthology.orchestra.run.trial040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:17.649 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:17.644+0000 7f5c13353780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-22T23:20:17.649 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:17.644+0000 7f5c13353780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-22T23:20:17.649 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:17.644+0000 7f5c13353780 -1 bdev(0x55ab19cc9800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:17.649 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:17.644+0000 7f5c13353780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-22T23:20:21.537 DEBUG:teuthology.orchestra.run.trial040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-22T23:20:21.601 DEBUG:teuthology.orchestra.run.trial040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:21.679 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:21.675+0000 7fa341827780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-22T23:20:21.679 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:21.675+0000 7fa341827780 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-22T23:20:21.679 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:21.675+0000 7fa341827780 -1 bdev(0x56519e531800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:21.680 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:21.676+0000 7fa341827780 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-22T23:20:25.568 DEBUG:teuthology.orchestra.run.trial040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-22T23:20:25.632 DEBUG:teuthology.orchestra.run.trial040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:25.711 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:25.707+0000 7fd2114d7780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2026-02-22T23:20:25.711 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:25.707+0000 7fd2114d7780 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2026-02-22T23:20:25.711 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:25.707+0000 7fd2114d7780 -1 bdev(0x562f66dd9800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:25.711 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:25.707+0000 7fd2114d7780 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2026-02-22T23:20:29.598 DEBUG:teuthology.orchestra.run.trial040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2026-02-22T23:20:29.661 DEBUG:teuthology.orchestra.run.trial040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:29.739 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:29.735+0000 7fce5a1f3780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2026-02-22T23:20:29.739 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:29.735+0000 7fce5a1f3780 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2026-02-22T23:20:29.739 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:29.735+0000 7fce5a1f3780 -1 bdev(0x55ad2c929800 /var/lib/ceph/osd/ceph-13/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:29.739 INFO:teuthology.orchestra.run.trial040.stderr:2026-02-22T23:20:29.735+0000 7fce5a1f3780 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2026-02-22T23:20:33.626 DEBUG:teuthology.orchestra.run.trial040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2026-02-22T23:20:33.691 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:33.718 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:33.718 INFO:tasks.ceph:role: osd.2 2026-02-22T23:20:33.719 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:20:33.719 DEBUG:teuthology.orchestra.run.trial080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T23:20:33.783 INFO:teuthology.orchestra.run.trial080.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:33.784 INFO:teuthology.orchestra.run.trial080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:34.353 INFO:teuthology.orchestra.run.trial080.stdout:Discarding blocks...Done. 2026-02-22T23:20:34.355 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial080.front.sepia.ceph.com -o noatime 2026-02-22T23:20:34.355 DEBUG:teuthology.orchestra.run.trial080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:34.429 DEBUG:teuthology.orchestra.run.trial080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:34.496 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2026-02-22T23:20:34.562 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:34.562 INFO:tasks.ceph:role: osd.6 2026-02-22T23:20:34.562 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:20:34.563 DEBUG:teuthology.orchestra.run.trial080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:34.627 INFO:teuthology.orchestra.run.trial080.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:34.628 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:34.628 INFO:teuthology.orchestra.run.trial080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:35.185 INFO:teuthology.orchestra.run.trial080.stdout:Discarding blocks...Done. 2026-02-22T23:20:35.187 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial080.front.sepia.ceph.com -o noatime 2026-02-22T23:20:35.187 DEBUG:teuthology.orchestra.run.trial080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2026-02-22T23:20:35.261 DEBUG:teuthology.orchestra.run.trial080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2026-02-22T23:20:35.331 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2026-02-22T23:20:35.400 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:35.400 INFO:tasks.ceph:role: osd.10 2026-02-22T23:20:35.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:20:35.401 DEBUG:teuthology.orchestra.run.trial080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:35.469 INFO:teuthology.orchestra.run.trial080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:36.061 INFO:teuthology.orchestra.run.trial080.stdout:Discarding blocks...Done. 2026-02-22T23:20:36.063 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial080.front.sepia.ceph.com -o noatime 2026-02-22T23:20:36.063 DEBUG:teuthology.orchestra.run.trial080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2026-02-22T23:20:36.130 DEBUG:teuthology.orchestra.run.trial080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2026-02-22T23:20:36.196 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2026-02-22T23:20:36.262 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:36.263 INFO:tasks.ceph:role: osd.14 2026-02-22T23:20:36.263 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:20:36.263 DEBUG:teuthology.orchestra.run.trial080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:36.329 INFO:teuthology.orchestra.run.trial080.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:36.330 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:36.330 INFO:teuthology.orchestra.run.trial080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:36.886 INFO:teuthology.orchestra.run.trial080.stdout:Discarding blocks...Done. 2026-02-22T23:20:36.888 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial080.front.sepia.ceph.com -o noatime 2026-02-22T23:20:36.888 DEBUG:teuthology.orchestra.run.trial080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2026-02-22T23:20:36.965 DEBUG:teuthology.orchestra.run.trial080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2026-02-22T23:20:37.039 DEBUG:teuthology.orchestra.run.trial080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:37.123 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:37.121+0000 7f802ae1e780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-22T23:20:37.123 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:37.121+0000 7f802ae1e780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-22T23:20:37.124 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:37.121+0000 7f802ae1e780 -1 bdev(0x557b7464f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:37.124 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:37.121+0000 7f802ae1e780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-22T23:20:41.010 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-22T23:20:41.076 DEBUG:teuthology.orchestra.run.trial080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:41.155 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:41.154+0000 7f88110d9780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2026-02-22T23:20:41.156 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:41.154+0000 7f88110d9780 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2026-02-22T23:20:41.156 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:41.154+0000 7f88110d9780 -1 bdev(0x5588c9711800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:41.156 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:41.154+0000 7f88110d9780 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2026-02-22T23:20:45.038 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2026-02-22T23:20:45.106 DEBUG:teuthology.orchestra.run.trial080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:45.186 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:45.185+0000 7ff6a0d16780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2026-02-22T23:20:45.187 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:45.185+0000 7ff6a0d16780 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2026-02-22T23:20:45.187 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:45.185+0000 7ff6a0d16780 -1 bdev(0x55ebe0831800 /var/lib/ceph/osd/ceph-10/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:45.187 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:45.185+0000 7ff6a0d16780 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2026-02-22T23:20:49.074 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2026-02-22T23:20:49.145 DEBUG:teuthology.orchestra.run.trial080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:49.227 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:49.225+0000 7f101be11780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2026-02-22T23:20:49.227 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:49.225+0000 7f101be11780 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2026-02-22T23:20:49.227 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:49.225+0000 7f101be11780 -1 bdev(0x559ddee29800 /var/lib/ceph/osd/ceph-14/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:49.228 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-22T23:20:49.225+0000 7f101be11780 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2026-02-22T23:20:53.106 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2026-02-22T23:20:53.172 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-22T23:20:53.198 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:53.198 INFO:tasks.ceph:role: osd.3 2026-02-22T23:20:53.198 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:20:53.198 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:53.265 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:53.835 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-22T23:20:53.837 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-22T23:20:53.837 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2026-02-22T23:20:53.905 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-22T23:20:53.972 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2026-02-22T23:20:54.036 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:54.036 INFO:tasks.ceph:role: osd.7 2026-02-22T23:20:54.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:20:54.036 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:54.103 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:54.663 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-22T23:20:54.665 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-22T23:20:54.665 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2026-02-22T23:20:54.735 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2026-02-22T23:20:54.801 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2026-02-22T23:20:54.867 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:54.867 INFO:tasks.ceph:role: osd.11 2026-02-22T23:20:54.867 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:20:54.867 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:54.931 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:55.490 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-22T23:20:55.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-22T23:20:55.492 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2026-02-22T23:20:55.559 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2026-02-22T23:20:55.626 DEBUG:teuthology.orchestra.run.trial189:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2026-02-22T23:20:55.696 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2026-02-22T23:20:55.696 INFO:tasks.ceph:role: osd.15 2026-02-22T23:20:55.696 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:20:55.696 DEBUG:teuthology.orchestra.run.trial189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:20:55.762 INFO:teuthology.orchestra.run.trial189.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:20:55.763 INFO:teuthology.orchestra.run.trial189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:20:56.321 INFO:teuthology.orchestra.run.trial189.stdout:Discarding blocks...Done. 2026-02-22T23:20:56.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial189.front.sepia.ceph.com -o noatime 2026-02-22T23:20:56.323 DEBUG:teuthology.orchestra.run.trial189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2026-02-22T23:20:56.390 DEBUG:teuthology.orchestra.run.trial189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2026-02-22T23:20:56.455 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:20:56.534 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:20:56.533+0000 7f3de6a15780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-22T23:20:56.534 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:20:56.534+0000 7f3de6a15780 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-22T23:20:56.535 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:20:56.534+0000 7f3de6a15780 -1 bdev(0x55e24ac2d800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-22T23:20:56.535 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:20:56.534+0000 7f3de6a15780 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-22T23:21:00.424 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-22T23:21:00.490 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:21:00.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:00.568+0000 7f5c3c52e780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2026-02-22T23:21:00.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:00.568+0000 7f5c3c52e780 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2026-02-22T23:21:00.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:00.568+0000 7f5c3c52e780 -1 bdev(0x55b14d763800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2026-02-22T23:21:00.568 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:00.568+0000 7f5c3c52e780 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2026-02-22T23:21:04.456 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2026-02-22T23:21:04.521 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:21:04.598 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:04.598+0000 7f03d400c780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2026-02-22T23:21:04.599 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:04.598+0000 7f03d400c780 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2026-02-22T23:21:04.599 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:04.598+0000 7f03d400c780 -1 bdev(0x555a81803800 /var/lib/ceph/osd/ceph-11/block) open stat got: (1) Operation not permitted 2026-02-22T23:21:04.599 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:04.598+0000 7f03d400c780 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2026-02-22T23:21:08.482 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2026-02-22T23:21:08.548 DEBUG:teuthology.orchestra.run.trial189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:21:08.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:08.626+0000 7fa20cfad780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2026-02-22T23:21:08.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:08.626+0000 7fa20cfad780 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2026-02-22T23:21:08.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:08.626+0000 7fa20cfad780 -1 bdev(0x562f6caab800 /var/lib/ceph/osd/ceph-15/block) open stat got: (1) Operation not permitted 2026-02-22T23:21:08.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:08.626+0000 7fa20cfad780 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2026-02-22T23:21:12.504 DEBUG:teuthology.orchestra.run.trial189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2026-02-22T23:21:12.569 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-22T23:21:12.569 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:12.569 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2026-02-22T23:21:12.595 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:12.595 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-22T23:21:12.659 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:12.659 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-22T23:21:12.721 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:12.721 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2026-02-22T23:21:12.784 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:12.785 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2026-02-22T23:21:12.851 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:12.851 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-22T23:21:12.884 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:12.884 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-22T23:21:12.948 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:12.948 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2026-02-22T23:21:13.012 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:13.012 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2026-02-22T23:21:13.076 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:13.076 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-22T23:21:13.102 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:13.103 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2026-02-22T23:21:13.168 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:13.168 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2026-02-22T23:21:13.231 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:13.232 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2026-02-22T23:21:13.297 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:13.297 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-22T23:21:13.321 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:13.322 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-22T23:21:13.386 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:13.386 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2026-02-22T23:21:13.450 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:13.450 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2026-02-22T23:21:13.514 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:13.514 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2026-02-22T23:21:13.577 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:13.577 DEBUG:teuthology.orchestra.run.trial189:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-22T23:21:13.633 INFO:tasks.ceph:Adding keys to all mons... 2026-02-22T23:21:13.633 DEBUG:teuthology.orchestra.run.trial006:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-22T23:21:13.635 DEBUG:teuthology.orchestra.run.trial040:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-22T23:21:13.638 DEBUG:teuthology.orchestra.run.trial080:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-22T23:21:13.656 INFO:teuthology.orchestra.run.trial006.stdout:[mgr.y] 2026-02-22T23:21:13.656 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAaj5tp79MbJBAAs1j5DBPIXgp+szQDv1k9JA== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.0] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAej5tp8pS4CxAAon5Y8qq3rsCemVSeUMroWw== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.4] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAij5tpVHRWDRAA1gOKE8IzcZgMJii8C74BKQ== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.8] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAmj5tphaCwDhAAXfGs1hVpwPOtzXxUk9bu5Q== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.12] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAqj5tpKJGREBAAEkykCwulA3hNQVvlWeiALw== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.1] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAxj5tpJpV1JhAAO97hdM8EWM69BjDZFLyCQQ== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.5] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQA1j5tpivJIKBAAmWMvBrFuVkGqO/9aRr8mrQ== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.9] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQA5j5tpqf8qKhAAkt0Zf+0Iw7L5w7v8H211Wg== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.13] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQA9j5tpNczhKxAAmETXy8kWeeMfIhXVg+WyjA== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.2] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBFj5tpFHRQBxAADQKNlBr6ZTfgE+iu7ndxxw== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.6] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBJj5tpCUFDCRAAZPTDjX/6RyaLjeqkEusl9w== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.10] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBNj5tpgF8eCxAAimM37XNfNp/1hz2iPnR4sA== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.14] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBRj5tp42V+DRAABmZVUr/KoZvwDEEe4g2DUg== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[mgr.x] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAaj5tplQ/ZJhAA1U1zYGuntPpWpxhSaWpd4A== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.3] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBYj5tpoRThHxAAnaf1SBkmhMLUyZs/gAkPdg== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.7] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBcj5tphCvqIRAAZobAHRXRxAxqHoZx4zGS4A== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.11] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBgj5tpC4e2IxAAlMgwEsKAZtjzXFdPG/WlkA== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[osd.15] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQBkj5tpDJRlJRAABqaKZg6h2zJ5uEmcDcDjbQ== 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout:[client.0] 2026-02-22T23:21:13.657 INFO:teuthology.orchestra.run.trial006.stdout: key = AQAaj5tpQ82BKxAAjjzaWj8iv3bhdgWaQXlVXg== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[mgr.y] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAaj5tp79MbJBAAs1j5DBPIXgp+szQDv1k9JA== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.0] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAej5tp8pS4CxAAon5Y8qq3rsCemVSeUMroWw== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.4] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAij5tpVHRWDRAA1gOKE8IzcZgMJii8C74BKQ== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.8] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAmj5tphaCwDhAAXfGs1hVpwPOtzXxUk9bu5Q== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.12] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAqj5tpKJGREBAAEkykCwulA3hNQVvlWeiALw== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.1] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAxj5tpJpV1JhAAO97hdM8EWM69BjDZFLyCQQ== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.5] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQA1j5tpivJIKBAAmWMvBrFuVkGqO/9aRr8mrQ== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.9] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQA5j5tpqf8qKhAAkt0Zf+0Iw7L5w7v8H211Wg== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.13] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQA9j5tpNczhKxAAmETXy8kWeeMfIhXVg+WyjA== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.2] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBFj5tpFHRQBxAADQKNlBr6ZTfgE+iu7ndxxw== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.6] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBJj5tpCUFDCRAAZPTDjX/6RyaLjeqkEusl9w== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.10] 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBNj5tpgF8eCxAAimM37XNfNp/1hz2iPnR4sA== 2026-02-22T23:21:13.659 INFO:teuthology.orchestra.run.trial040.stdout:[osd.14] 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBRj5tp42V+DRAABmZVUr/KoZvwDEEe4g2DUg== 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout:[mgr.x] 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAaj5tplQ/ZJhAA1U1zYGuntPpWpxhSaWpd4A== 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout:[osd.3] 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBYj5tpoRThHxAAnaf1SBkmhMLUyZs/gAkPdg== 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout:[osd.7] 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBcj5tphCvqIRAAZobAHRXRxAxqHoZx4zGS4A== 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout:[osd.11] 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBgj5tpC4e2IxAAlMgwEsKAZtjzXFdPG/WlkA== 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout:[osd.15] 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout: key = AQBkj5tpDJRlJRAABqaKZg6h2zJ5uEmcDcDjbQ== 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout:[client.0] 2026-02-22T23:21:13.660 INFO:teuthology.orchestra.run.trial040.stdout: key = AQAaj5tpQ82BKxAAjjzaWj8iv3bhdgWaQXlVXg== 2026-02-22T23:21:13.661 INFO:teuthology.orchestra.run.trial080.stdout:[mgr.y] 2026-02-22T23:21:13.661 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAaj5tp79MbJBAAs1j5DBPIXgp+szQDv1k9JA== 2026-02-22T23:21:13.661 INFO:teuthology.orchestra.run.trial080.stdout:[osd.0] 2026-02-22T23:21:13.661 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAej5tp8pS4CxAAon5Y8qq3rsCemVSeUMroWw== 2026-02-22T23:21:13.661 INFO:teuthology.orchestra.run.trial080.stdout:[osd.4] 2026-02-22T23:21:13.661 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAij5tpVHRWDRAA1gOKE8IzcZgMJii8C74BKQ== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.8] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAmj5tphaCwDhAAXfGs1hVpwPOtzXxUk9bu5Q== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.12] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAqj5tpKJGREBAAEkykCwulA3hNQVvlWeiALw== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.1] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAxj5tpJpV1JhAAO97hdM8EWM69BjDZFLyCQQ== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.5] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQA1j5tpivJIKBAAmWMvBrFuVkGqO/9aRr8mrQ== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.9] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQA5j5tpqf8qKhAAkt0Zf+0Iw7L5w7v8H211Wg== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.13] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQA9j5tpNczhKxAAmETXy8kWeeMfIhXVg+WyjA== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.2] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBFj5tpFHRQBxAADQKNlBr6ZTfgE+iu7ndxxw== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.6] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBJj5tpCUFDCRAAZPTDjX/6RyaLjeqkEusl9w== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.10] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBNj5tpgF8eCxAAimM37XNfNp/1hz2iPnR4sA== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.14] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBRj5tp42V+DRAABmZVUr/KoZvwDEEe4g2DUg== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[mgr.x] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAaj5tplQ/ZJhAA1U1zYGuntPpWpxhSaWpd4A== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.3] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBYj5tpoRThHxAAnaf1SBkmhMLUyZs/gAkPdg== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.7] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBcj5tphCvqIRAAZobAHRXRxAxqHoZx4zGS4A== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.11] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBgj5tpC4e2IxAAlMgwEsKAZtjzXFdPG/WlkA== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[osd.15] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBkj5tpDJRlJRAABqaKZg6h2zJ5uEmcDcDjbQ== 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout:[client.0] 2026-02-22T23:21:13.662 INFO:teuthology.orchestra.run.trial080.stdout: key = AQAaj5tpQ82BKxAAjjzaWj8iv3bhdgWaQXlVXg== 2026-02-22T23:21:13.663 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:21:13.700 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:21:13.703 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:21:13.743 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.782 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.785 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.827 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.862 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.865 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.907 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.944 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.947 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:13.987 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.023 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.025 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.067 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.102 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.106 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.147 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.181 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.184 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.227 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.261 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.264 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.307 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.340 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.343 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.388 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.420 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.423 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.467 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.503 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.505 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.547 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.582 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.584 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.625 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.662 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.665 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.704 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:21:14.742 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:21:14.745 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:21:14.788 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.822 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.825 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.867 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.904 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.907 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.948 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.984 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:14.987 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:15.027 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:15.074 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:15.081 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:21:15.122 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-22T23:21:15.154 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-22T23:21:15.158 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-22T23:21:15.203 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-22T23:21:15.203 DEBUG:teuthology.orchestra.run.trial006:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-22T23:21:15.255 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T23:21:15.340 DEBUG:teuthology.orchestra.run.trial006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-22T23:21:15.364 DEBUG:teuthology.orchestra.run.trial040:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-22T23:21:15.389 DEBUG:teuthology.orchestra.run.trial040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T23:21:15.477 DEBUG:teuthology.orchestra.run.trial040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-22T23:21:15.502 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-22T23:21:15.528 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T23:21:15.617 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-22T23:21:15.641 DEBUG:teuthology.orchestra.run.trial006:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:21:15.644 DEBUG:teuthology.orchestra.run.trial040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:21:15.646 DEBUG:teuthology.orchestra.run.trial080:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:21:15.697 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-22T23:21:15.697 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-22T23:21:15.698 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-22T23:21:15.698 DEBUG:teuthology.orchestra.run.trial006:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2026-02-22T23:21:15.701 INFO:tasks.ceph.mon.a:Started 2026-02-22T23:21:15.702 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-22T23:21:15.702 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-22T23:21:15.702 DEBUG:teuthology.orchestra.run.trial040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2026-02-22T23:21:15.704 INFO:tasks.ceph.mon.b:Started 2026-02-22T23:21:15.705 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-22T23:21:15.705 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-22T23:21:15.705 DEBUG:teuthology.orchestra.run.trial080:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2026-02-22T23:21:15.739 INFO:tasks.ceph.mon.c:Started 2026-02-22T23:21:15.739 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-22T23:21:15.739 INFO:tasks.ceph.mgr.y:Restarting daemon 2026-02-22T23:21:15.739 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2026-02-22T23:21:15.742 INFO:tasks.ceph.mgr.y:Started 2026-02-22T23:21:15.743 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-22T23:21:15.743 DEBUG:teuthology.orchestra.run.trial189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2026-02-22T23:21:15.746 INFO:tasks.ceph.mgr.x:Started 2026-02-22T23:21:15.746 INFO:tasks.ceph:Setting crush tunables to default 2026-02-22T23:21:15.746 DEBUG:teuthology.orchestra.run.trial006:> sudo ceph --cluster ceph osd crush tunables default 2026-02-22T23:21:15.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.811+0000 7f84bdad6640 10 monclient: get_monmap_and_config 2026-02-22T23:21:15.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.811+0000 7f84bdad6640 10 monclient: build_initial_monmap 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.811+0000 7f84bdad6640 10 monclient: monmap: 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:21:15.813318+0000 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:21:15.813318+0000 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:15.814 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 1 Processor -- start 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 1 -- start start 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient: init 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient: _add_conns ranks=[0,4,2,3,5,1] 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f84b8155ef0 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 0x7f84b815c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient(hunting): picked mon.noname-b con 0x7f84b81562d0 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f84b80cde80 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce3b0 con 0x7f84b8155ef0 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f84b8155ef0 legacy=0x7f84b80cb620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce690 con 0x7f84b80cde80 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f84b81562d0 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:15.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.812+0000 7f84bdad6640 10 monclient(hunting): authenticate will time out at 950.540925s 2026-02-22T23:21:15.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.816+0000 7f84b7fff640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f84b80cde80 legacy=0x7f84b80c9080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:15.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.816+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 msgr2=0x7f84b815c870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:15.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:15.816+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 0x7f84b815c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:16.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.013+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f84b8155ef0 legacy=0x7f84b80cb620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:16.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.017+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 msgr2=0x7f84b815c870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:16.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.017+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 0x7f84b815c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:16.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.017+0000 7f84b7fff640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f84b80cde80 legacy=0x7f84b80c9080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:16.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.414+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f84b8155ef0 legacy=0x7f84b80cb620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:16.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.418+0000 7f84b7fff640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f84b80cde80 legacy=0x7f84b80c9080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:16.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.418+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 msgr2=0x7f84b815c870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:16.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:16.418+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 0x7f84b815c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:17.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:17.215+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f84b8155ef0 legacy=0x7f84b80cb620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:17.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:17.219+0000 7f84b7fff640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f84b80cde80 legacy=0x7f84b80c9080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:17.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:17.219+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 msgr2=0x7f84b815c870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:17.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:17.219+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 0x7f84b815c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:18.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): tick 2026-02-22T23:21:18.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): _check_auth_tickets 2026-02-22T23:21:18.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 monclient(hunting): continuing hunt 2026-02-22T23:21:18.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-22T23:21:18.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 msgr2=0x7f84b815c870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:18.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f84b81562d0 0x7f84b815c870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:18.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f84b80cde80 legacy=0x7f84b80c9080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f84b8155ef0 legacy=0x7f84b80cb620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient: _add_conns ranks=[3,5,1] 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 0x7f8494005c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-a con 0x7f84940037d0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 0x7f8494008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-c con 0x7f8494006270 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8494008e10 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce970 con 0x7f8494008e10 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f84940037d0 2026-02-22T23:21:18.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:18.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f849400dc50 con 0x7f8494006270 2026-02-22T23:21:18.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b57fa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:18.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 msgr2=0x7f8494005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:18.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.812+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 0x7f8494005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:18.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.813+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494008e10 legacy=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:18.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.813+0000 7f84b6ffd640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 msgr2=0x7f8494008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:18.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:18.813+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 0x7f8494008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:19.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.014+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 msgr2=0x7f8494005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:19.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.014+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 0x7f8494005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:19.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.014+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494008e10 legacy=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:19.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.014+0000 7f84b6ffd640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 msgr2=0x7f8494008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:19.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.014+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 0x7f8494008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:19.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.415+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 msgr2=0x7f8494005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:19.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.415+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 0x7f8494005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:19.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.415+0000 7f84b6ffd640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 msgr2=0x7f8494008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:19.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.415+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 0x7f8494008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:19.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:19.415+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494008e10 legacy=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:20.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:20.216+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 msgr2=0x7f8494005c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:20.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:20.216+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 0x7f8494005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:20.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:20.216+0000 7f84b6ffd640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 msgr2=0x7f8494008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:20.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:20.216+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494008e10 legacy=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:20.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:20.216+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 0x7f8494008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:21.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): tick 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): _check_auth_tickets 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 monclient(hunting): continuing hunt 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 msgr2=0x7f8494008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8494006270 0x7f8494008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 msgr2=0x7f8494005c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940037d0 0x7f8494005c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494008e10 legacy=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b7fff640 1 -- reap_dead start 2026-02-22T23:21:21.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient: _add_conns ranks=[0,1,4,2,3,5] 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b7fff640 1 -- reap_dead start 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f849400cc50 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f849400d7e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 0x7f8494007e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-b con 0x7f8494005ce0 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce3b0 con 0x7f849400cc50 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce970 con 0x7f849400d7e0 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f8494005ce0 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b57fa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400cc50 legacy=0x7f849400d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:21.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b6ffd640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f849400d7e0 legacy=0x7f8494007730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:21.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b7fff640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 msgr2=0x7f8494007e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:21.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:21.814+0000 7f84b7fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 0x7f8494007e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:22.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.015+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400cc50 legacy=0x7f849400d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:22.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.015+0000 7f84b6ffd640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f849400d7e0 legacy=0x7f8494007730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:22.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.015+0000 7f84b7fff640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 msgr2=0x7f8494007e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:22.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.015+0000 7f84b7fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 0x7f8494007e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:22.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.416+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400cc50 legacy=0x7f849400d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:22.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.416+0000 7f84b6ffd640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f849400d7e0 legacy=0x7f8494007730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:22.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.416+0000 7f84b7fff640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 msgr2=0x7f8494007e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:22.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:22.416+0000 7f84b7fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 0x7f8494007e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:23.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:23.217+0000 7f84b77fe640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400cc50 legacy=0x7f849400d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:23.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:23.217+0000 7f84b6ffd640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f849400d7e0 legacy=0x7f8494007730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:23.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:23.217+0000 7f84b7fff640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 msgr2=0x7f8494007e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:23.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:23.217+0000 7f84b7fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 0x7f8494007e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:24.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): tick 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): _check_auth_tickets 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 monclient(hunting): continuing hunt 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 msgr2=0x7f8494007e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494005ce0 0x7f8494007e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f849400d7e0 legacy=0x7f8494007730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400cc50 legacy=0x7f849400d030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient: _add_conns ranks=[3,2,5] 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 0x7f849400b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:24.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-a con 0x7f8494008520 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f849400ba80 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 0x7f8494017fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-c con 0x7f849400c5a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce690 con 0x7f849400ba80 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f8494008520 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f849400dc50 con 0x7f849400c5a0 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b57fa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 msgr2=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 0x7f849400b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:24.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f849400ba80 legacy=0x7f849400be90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:24.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b7fff640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 msgr2=0x7f8494017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:24.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:24.814+0000 7f84b7fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 0x7f8494017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.015+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 msgr2=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.015+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 0x7f849400b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.015+0000 7f84b7fff640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 msgr2=0x7f8494017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.015+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f849400ba80 legacy=0x7f849400be90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.015+0000 7f84b7fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 0x7f8494017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:25.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.416+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 msgr2=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:25.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.416+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 0x7f849400b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:25.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.416+0000 7f84b7fff640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 msgr2=0x7f8494017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:25.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.416+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f849400ba80 legacy=0x7f849400be90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:25.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:25.416+0000 7f84b7fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 0x7f8494017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:26.198 INFO:tasks.ceph.mon.b.trial040.stderr:--00:00:00:10.444 44392-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-22T23:21:26.214 INFO:tasks.ceph.mon.c.trial080.stderr:--00:00:00:10.425 44468-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-22T23:21:26.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:26.217+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 msgr2=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:26.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:26.217+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 0x7f849400b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:26.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:26.217+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f849400ba80 legacy=0x7f849400be90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:26.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:26.217+0000 7f84b7fff640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 msgr2=0x7f8494017fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:26.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:26.217+0000 7f84b7fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 0x7f8494017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:26.258 INFO:tasks.ceph.mon.a.trial006.stderr:--00:00:00:10.508 44699-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-22T23:21:27.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): tick 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): _check_auth_tickets 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 monclient(hunting): continuing hunt 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 msgr2=0x7f8494017fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400c5a0 0x7f8494017fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 msgr2=0x7f849400b2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8494008520 0x7f849400b2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f849400ba80 legacy=0x7f849400be90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b7fff640 1 -- reap_dead start 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient: _add_conns ranks=[3,2,0,5,4,1] 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 0x7f849400cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-a con 0x7f84940084c0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8494005bc0 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f849400b2b0 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce3b0 con 0x7f849400b2b0 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce690 con 0x7f8494005bc0 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f84940084c0 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b57fa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b7fff640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400b2b0 legacy=0x7f849400c190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 0x7f849400cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:27.814+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f8494005bc0 legacy=0x7f849400ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:28.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.015+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:28.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.015+0000 7f84b7fff640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400b2b0 legacy=0x7f849400c190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:28.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.015+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 0x7f849400cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:28.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.015+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f8494005bc0 legacy=0x7f849400ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:28.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.416+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:28.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.416+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 0x7f849400cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:28.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.416+0000 7f84b7fff640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400b2b0 legacy=0x7f849400c190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:28.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:28.416+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f8494005bc0 legacy=0x7f849400ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:29.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:29.217+0000 7f84b77fe640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400cc50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.6:3300/0 2026-02-22T23:21:29.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:29.217+0000 7f84b7fff640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400b2b0 legacy=0x7f849400c190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.6:6789/0 2026-02-22T23:21:29.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:29.217+0000 7f84b77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 0x7f849400cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:29.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:29.217+0000 7f84b6ffd640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f8494005bc0 legacy=0x7f849400ba80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.80:6789/0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): tick 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): _check_auth_tickets 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 monclient(hunting): continuing hunt 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400cc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f84940084c0 0x7f849400cc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 -- >> v1:10.20.193.80:6789/0 conn(0x7f8494005bc0 legacy=0x7f849400ba80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 -- >> v1:10.20.193.6:6789/0 conn(0x7f849400b2b0 legacy=0x7f849400c190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient: _add_conns ranks=[5,4,1] 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 0x7f84940076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-c con 0x7f849400d190 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 0x7f8494014fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-b con 0x7f8494007bf0 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8494007ff0 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce970 con 0x7f8494007ff0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f8494007bf0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f849400dc50 con 0x7f849400d190 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b57fa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494007ff0 legacy=0x7f8494017510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 msgr2=0x7f8494014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 0x7f8494014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b77fe640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:30.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:30.814+0000 7f84b77fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.015+0000 7f84b77fe640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:31.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.015+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 msgr2=0x7f8494014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:31.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.015+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494007ff0 legacy=0x7f8494017510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:31.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.015+0000 7f84b77fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:31.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.015+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 0x7f8494014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:31.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.416+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 msgr2=0x7f8494014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:31.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.416+0000 7f84b77fe640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:31.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.416+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494007ff0 legacy=0x7f8494017510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:31.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.416+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 0x7f8494014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:31.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:31.416+0000 7f84b77fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:31.759 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:00:15.970 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:21:31.762 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:00:16.008 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:21:31.819 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:00:16.069 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:21:31.910 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:00:16.121 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:21:31.913 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:00:16.159 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:21:31.971 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:00:16.221 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:21:32.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:32.217+0000 7f84b6ffd640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 msgr2=0x7f8494014fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.40:3300/0 2026-02-22T23:21:32.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:32.217+0000 7f84b77fe640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:10.20.193.80:3300/0 2026-02-22T23:21:32.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:32.217+0000 7f84b7fff640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494007ff0 legacy=0x7f8494017510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:10.20.193.40:6789/0 2026-02-22T23:21:32.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:32.217+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 0x7f8494014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:32.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:32.217+0000 7f84b77fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:33.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): tick 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): _check_auth_tickets 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 monclient(hunting): continuing hunt 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f849400d190 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 -- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 msgr2=0x7f8494014fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8494007bf0 0x7f8494014fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 -- >> v1:10.20.193.40:6789/0 conn(0x7f8494007ff0 legacy=0x7f8494017510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b7fff640 1 -- reap_dead start 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b7fff640 1 -- reap_dead start 2026-02-22T23:21:33.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient: _add_conns ranks=[0,5,3,1,4,2] 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8494007ff0 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-c con 0x7f84940084c0 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-a con 0x7f8494005bc0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce3b0 con 0x7f8494007ff0 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f8494005bc0 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f849400dc50 con 0x7f84940084c0 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.814+0000 7f84b57fa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:33.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:39338/0 (socket says 10.20.193.6:39338) 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 learned_addr learned my addr 10.20.193.6:0/1152479734 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request con 0x7f84940084c0 auth_method 0 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:33.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.815+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:33.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.827+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:33.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.827+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f8494005bc0 auth_method 0 2026-02-22T23:21:33.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.827+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:33.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.827+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:33.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.827+0000 7f84b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:33.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.828+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-22T23:21:33.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.828+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:33.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.828+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:33.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:33.828+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:34.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.016+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:34.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.016+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request con 0x7f84940084c0 auth_method 0 2026-02-22T23:21:34.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.016+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:34.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.016+0000 7f84b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:34.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.016+0000 7f84b6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:34.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.017+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-22T23:21:34.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.017+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:34.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.017+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:34.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.017+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f8494005bc0 auth_method 0 2026-02-22T23:21:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-22T23:21:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.029+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:34.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.418+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.418+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request con 0x7f84940084c0 auth_method 0 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.418+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.418+0000 7f84b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.418+0000 7f84b6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.419+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.419+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.419+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.419+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f8494005bc0 auth_method 0 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:34.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:34.431+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:35.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.220+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:35.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.220+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request con 0x7f84940084c0 auth_method 0 2026-02-22T23:21:35.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.220+0000 7f84b6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:35.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.220+0000 7f84b6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:35.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.220+0000 7f84b6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:35.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.221+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-22T23:21:35.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.221+0000 7f84b6ffd640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:35.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.221+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:35.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.221+0000 7f84b6ffd640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:35.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.232+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:35.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f8494005bc0 auth_method 0 2026-02-22T23:21:35.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:35.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:35.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:35.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2026-02-22T23:21:35.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:35.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:35.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:35.233+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:36.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): tick 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): _check_auth_tickets 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 monclient(hunting): continuing hunt 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): _reopen_session rank -1 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 msgr2=0x7f849400b270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.80:3300/0 conn(0x7f84940084c0 0x7f849400b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 msgr2=0x7f84940076b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.6:3300/0 conn(0x7f8494005bc0 0x7f84940076b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 -- 10.20.193.6:0/1152479734 >> v1:10.20.193.6:6789/0 conn(0x7f8494007ff0 legacy=0x7f849400cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient: _add_conns ranks=[2,1,4] 2026-02-22T23:21:36.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8494007bf0 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f849400c190 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): picked mon.noname-b con 0x7f84940166a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 -- 10.20.193.6:0/1152479734 --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce970 con 0x7f849400c190 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 -- 10.20.193.6:0/1152479734 --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84b80ce690 con 0x7f8494007bf0 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 1 -- 10.20.193.6:0/1152479734 --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f84940166a0 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.814+0000 7f84b57fa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.815+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.815+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f84940166a0 auth_method 0 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.815+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:36.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.815+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:36.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.815+0000 7f84b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:36.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.816+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-22T23:21:36.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.816+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:36.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.816+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:36.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:36.816+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:37.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:37.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f84940166a0 auth_method 0 2026-02-22T23:21:37.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:37.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:37.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:37.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-22T23:21:37.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:37.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:37.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.017+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2026-02-22T23:21:37.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.418+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.418+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f84940166a0 auth_method 0 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.418+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.418+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.418+0000 7f84b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.419+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.419+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.419+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:37.419+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2026-02-22T23:21:38.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:38.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f84940166a0 auth_method 0 2026-02-22T23:21:38.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:38.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:38.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:38.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-22T23:21:38.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:38.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.220+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:38.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:38.221+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2026-02-22T23:21:39.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.360+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429314900 0 0) 0x7f84b80ce970 con 0x7f849400c190 2026-02-22T23:21:39.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.361+0000 7f84b67fc640 10 monclient(hunting): _negotiate 2026-02-22T23:21:39.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.361+0000 7f84b67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:39.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.361+0000 7f84b67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:39.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.361+0000 7f84b67fc640 10 monclient(hunting): my global_id is 4102 2026-02-22T23:21:39.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.361+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f849c003620 con 0x7f849400c190 2026-02-22T23:21:39.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.376+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1633740275 0 0) 0x7f84b80ce690 con 0x7f8494007bf0 2026-02-22T23:21:39.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.376+0000 7f84b67fc640 10 monclient(hunting): _negotiate 2026-02-22T23:21:39.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.376+0000 7f84b67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:39.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.376+0000 7f84b67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:39.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.376+0000 7f84b67fc640 10 monclient(hunting): my global_id is 4103 2026-02-22T23:21:39.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.376+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84b80ce970 con 0x7f8494007bf0 2026-02-22T23:21:39.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4177225284 0 0) 0x7f849c003620 con 0x7f849400c190 2026-02-22T23:21:39.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 msgr2=0x7f8494016ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:39.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 --2- 10.20.193.6:0/1152479734 >> v2:10.20.193.40:3300/0 conn(0x7f84940166a0 0x7f8494016ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 >> v1:10.20.193.80:6789/0 conn(0x7f8494007bf0 legacy=0x7f849400ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: _finish_hunting 0 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 reap_dead start 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849400c5a0 con 0x7f849400c190 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: _finish_auth 0 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:39.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f84a8009840 con 0x7f849400c190 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: dump: 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:39.423 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 >> v1:10.20.193.40:6789/0 conn(0x7f849400c190 legacy=0x7f8494014f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c003c90 0x7f849c004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): picked mon.b con 0x7f849c003c90 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849c004900 0x7f849c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): picked mon.c con 0x7f849c004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 0x7f849c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:39.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): picked mon.a con 0x7f849c005430 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f84b815cdb0 con 0x7f849c005430 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f849c00c380 con 0x7f849c003c90 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f849c00c580 con 0x7f849c004900 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b67fc640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.418+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 0x7f849c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.419+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f849c005430 auth_method 0 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.419+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.419+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:39.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.419+0000 7f84b7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:39.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b77fe640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c003c90 0x7f849c004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:39.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:39.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:39.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:21:39.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b77fe640 10 monclient(hunting): get_auth_request con 0x7f849c003c90 auth_method 0 2026-02-22T23:21:39.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:39.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 msgr2=0x7f849c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b7fff640 1 -- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 msgr2=0x7f849c00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 0x7f849c00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.423+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 0x7f849c00bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.425+0000 7f84b77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.425+0000 7f84b77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:39.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.425+0000 7f84b77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:21:39.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 10 monclient(hunting): handle_auth_done global_id 4102 payload 293 2026-02-22T23:21:39.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 1 -- 10.20.193.6:0/1152479734 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849c004900 msgr2=0x7f849c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:39.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849c004900 0x7f849c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 1 -- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 msgr2=0x7f849c00bac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:21:39.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 0x7f849c00bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 1 monclient: found mon.b 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 1 -- 10.20.193.6:0/1152479734 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c003620 con 0x7f849c003c90 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 10 monclient: _finish_auth 0 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84b77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84bdad6640 5 monclient: authenticate success, global_id 4102 2026-02-22T23:21:39.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.430+0000 7f84bdad6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:39.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.432+0000 7f84b77fe640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c003c90 0x7f849c004090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8494008a40 tx=0x7f84a402f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6767eb569178fcd1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:39.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a40099f0 con 0x7f849c003c90 2026-02-22T23:21:39.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:39.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:39.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 10 monclient: dump: 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a4033070 con 0x7f849c003c90 2026-02-22T23:21:39.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 1 -- 10.20.193.6:0/1152479734 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a40386b0 con 0x7f849c003c90 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84bdad6640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84bdad6640 4 set_mon_vals no callback set 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.451+0000 7f84b67fc640 10 monclient: dump: 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:39.457 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 10 monclient: shutdown 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 -- 10.20.193.6:0/1152479734 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c003c90 msgr2=0x7f849c004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c003c90 0x7f849c004090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8494008a40 tx=0x7f84a402f410 comp rx=0 tx=0).stop 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 -- 10.20.193.6:0/1152479734 shutdown_connections 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c005430 0x7f849c00bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849c004900 0x7f849c004d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1152479734 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c003c90 0x7f849c004090 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 -- 10.20.193.6:0/1152479734 >> 10.20.193.6:0/1152479734 conn(0x7f84b806d8c0 msgr2=0x7f84b806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 -- 10.20.193.6:0/1152479734 shutdown_connections 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 1 -- 10.20.193.6:0/1152479734 wait complete. 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 10 monclient: build_initial_monmap 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.452+0000 7f84bdad6640 10 monclient: monmap: 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:39.459 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 librados: starting msgr at 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 librados: starting objecter 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 Processor -- start 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 -- start start 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 librados: setting wanted keys 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 librados: calling monclient init 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient: init 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849400c190 0x7f84b8148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:39.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): picked mon.noname-c con 0x7f849400c190 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c003c90 0x7f84b80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): picked mon.noname-a con 0x7f849c003c90 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c004900 0x7f84b8145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): picked mon.noname-b con 0x7f849c004900 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f84b80cef30 con 0x7f849c003c90 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f84b80d2730 con 0x7f849c004900 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f84b815c870 con 0x7f849400c190 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:39.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.453+0000 7f84bdad6640 10 monclient(hunting): authenticate will time out at 974.180655s 2026-02-22T23:21:39.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c004900 0x7f84b8145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c004900 0x7f84b8145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:38026/0 (socket says 10.20.193.6:38026) 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 1 -- 10.20.193.6:0/1284088514 learned_addr learned my addr 10.20.193.6:0/1284088514 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 10 monclient(hunting): get_auth_request con 0x7f849c004900 auth_method 0 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:39.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.457+0000 7f84b7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:39.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 10 monclient(hunting): handle_auth_done global_id 4105 payload 824 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 1 -- 10.20.193.6:0/1284088514 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849400c190 msgr2=0x7f84b8148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1284088514 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849400c190 0x7f84b8148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 1 -- 10.20.193.6:0/1284088514 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c003c90 msgr2=0x7f84b80cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1284088514 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c003c90 0x7f84b80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 1 monclient: found mon.noname-b 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:39.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.463+0000 7f84b7fff640 1 -- 10.20.193.6:0/1284088514 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b8147650 con 0x7f849c004900 2026-02-22T23:21:39.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.465+0000 7f84b7fff640 1 --2- 10.20.193.6:0/1284088514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c004900 0x7f84b8145810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f849400cb50 tx=0x7f84ac016f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:39.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.465+0000 7f8493fff640 10 client.?.objecter ms_handle_connect 0x7f849c004900 2026-02-22T23:21:39.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.465+0000 7f8493fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:39.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 1 -- 10.20.193.6:0/1284088514 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac025070 con 0x7f849c004900 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: dump: 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:39.473 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: _finish_auth 0 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: _check_auth_tickets 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 1 -- 10.20.193.6:0/1284088514 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ac013d00 con 0x7f849c004900 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 1 -- 10.20.193.6:0/1284088514 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac016250 con 0x7f849c004900 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f8493fff640 10 monclient: dump: 2026-02-22T23:21:39.474 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84b5ffb640 4 set_mon_vals no callback set 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 5 monclient: authenticate success, global_id 4105 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 10 monclient: _renew_subs 2026-02-22T23:21:39.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b8149380 con 0x7f849c004900 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 10 client.4105.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 10 monclient: _renew_subs 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b8149940 con 0x7f849c004900 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.470+0000 7f84bdad6640 1 librados: init done 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.471+0000 7f84bdad6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.471+0000 7f84bdad6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.471+0000 7f84bdad6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:39.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.471+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b80cb620 con 0x7f849c004900 2026-02-22T23:21:39.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.560+0000 7f8493fff640 1 -- 10.20.193.6:0/1284088514 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f84ac014440 con 0x7f849c004900 2026-02-22T23:21:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.564+0000 7f8493fff640 1 -- 10.20.193.6:0/1284088514 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f84ac020a90 con 0x7f849c004900 2026-02-22T23:21:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.564+0000 7f8493fff640 10 client.4105.objecter ms_dispatch2 0x7f84b8000b90 osd_map(1..1 src has 1..1) 2026-02-22T23:21:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.564+0000 7f8493fff640 3 client.4105.objecter handle_osd_map got epochs [1,1] > 0 2026-02-22T23:21:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.564+0000 7f8493fff640 3 client.4105.objecter handle_osd_map decoding full epoch 1 2026-02-22T23:21:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.564+0000 7f8493fff640 20 client.4105.objecter dump_active .. 0 homeless 2026-02-22T23:21:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.565+0000 7f8493fff640 1 -- 10.20.193.6:0/1284088514 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f84ac02e060 con 0x7f849c004900 2026-02-22T23:21:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.565+0000 7f8493fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.565+0000 7f8493fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:39.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.595+0000 7f84bdad6640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-22T23:21:39.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.595+0000 7f84bdad6640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-22T23:21:39.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.595+0000 7f84bdad6640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:39.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.595+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f84b80c9670 con 0x7f849c004900 2026-02-22T23:21:39.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.739+0000 7f8493fff640 1 -- 10.20.193.6:0/1284088514 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f84b80c9670 con 0x7f849c004900 2026-02-22T23:21:39.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.739+0000 7f8493fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2026-02-22T23:21:39.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.739+0000 7f8493fff640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2026-02-22T23:21:39.741 INFO:teuthology.orchestra.run.trial006.stderr:adjusted tunables profile to default 2026-02-22T23:21:39.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 10 librados: watch_flush enter 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 10 librados: watch_flush exit 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 20 client.4105.objecter shutdown clearing up homeless session... 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 10 client.4105.objecter successfully canceled tick 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 10 monclient: shutdown 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c004900 msgr2=0x7f84b8145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1284088514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c004900 0x7f84b8145810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f849400cb50 tx=0x7f84ac016f60 comp rx=0 tx=0).stop 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 shutdown_connections 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1284088514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f849c004900 0x7f84b8145810 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1284088514 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f849c003c90 0x7f84b80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 --2- 10.20.193.6:0/1284088514 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f849400c190 0x7f84b8148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:39.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 >> 10.20.193.6:0/1284088514 conn(0x7f84b806d8c0 msgr2=0x7f84b80c9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:39.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 shutdown_connections 2026-02-22T23:21:39.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 -- 10.20.193.6:0/1284088514 wait complete. 2026-02-22T23:21:39.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:39.740+0000 7f84bdad6640 1 librados: shutdown 2026-02-22T23:21:39.750 INFO:tasks.ceph:check_enable_crimson: False 2026-02-22T23:21:39.750 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-22T23:21:39.750 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:39.750 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-22T23:21:39.776 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:39.776 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-22T23:21:39.842 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:39.842 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2026-02-22T23:21:39.872 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:39.872+0000 7f73757cf140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-22T23:21:39.906 DEBUG:teuthology.orchestra.run.trial006:> set -ex 2026-02-22T23:21:39.906 DEBUG:teuthology.orchestra.run.trial006:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2026-02-22T23:21:39.971 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:39.971 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-22T23:21:39.993 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:39.993+0000 7f73757cf140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-22T23:21:39.998 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:39.998 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-22T23:21:40.021 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.021+0000 7f73757cf140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-22T23:21:40.046 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.044+0000 7f324ffd8140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-22T23:21:40.054 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.054+0000 7f73757cf140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-22T23:21:40.062 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:40.062 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2026-02-22T23:21:40.077 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.077+0000 7f73757cf140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-22T23:21:40.103 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.103+0000 7f73757cf140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-22T23:21:40.126 DEBUG:teuthology.orchestra.run.trial040:> set -ex 2026-02-22T23:21:40.126 DEBUG:teuthology.orchestra.run.trial040:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2026-02-22T23:21:40.159 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.159+0000 7f73757cf140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-22T23:21:40.166 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.165+0000 7f324ffd8140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-22T23:21:40.190 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:40.190 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-22T23:21:40.193 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.192+0000 7f324ffd8140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-22T23:21:40.217 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:40.218 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2026-02-22T23:21:40.221 INFO:tasks.ceph.mgr.x.trial189.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-22T23:21:40.221 INFO:tasks.ceph.mgr.x.trial189.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-22T23:21:40.221 INFO:tasks.ceph.mgr.x.trial189.stderr: from numpy import show_config as show_numpy_config 2026-02-22T23:21:40.224 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.224+0000 7f73757cf140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-22T23:21:40.226 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.225+0000 7f324ffd8140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-22T23:21:40.249 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.248+0000 7f324ffd8140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-22T23:21:40.249 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.249+0000 7f73757cf140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-22T23:21:40.276 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.274+0000 7f324ffd8140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-22T23:21:40.282 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:40.282 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2026-02-22T23:21:40.305 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.305+0000 7f73757cf140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-22T23:21:40.333 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.331+0000 7f324ffd8140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-22T23:21:40.346 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-22T23:21:40.347 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2026-02-22T23:21:40.347 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.347+0000 7f73757cf140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-22T23:21:40.373 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.373+0000 7f73757cf140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-22T23:21:40.394 INFO:tasks.ceph.mgr.y.trial006.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-22T23:21:40.394 INFO:tasks.ceph.mgr.y.trial006.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-22T23:21:40.394 INFO:tasks.ceph.mgr.y.trial006.stderr: from numpy import show_config as show_numpy_config 2026-02-22T23:21:40.397 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.396+0000 7f324ffd8140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-22T23:21:40.398 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.398+0000 7f73757cf140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-22T23:21:40.412 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:40.412 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-22T23:21:40.423 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.421+0000 7f324ffd8140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-22T23:21:40.424 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.423+0000 7f73757cf140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-22T23:21:40.437 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:40.437 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2026-02-22T23:21:40.446 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.446+0000 7f73757cf140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-22T23:21:40.470 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:40.470 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2026-02-22T23:21:40.478 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.476+0000 7f324ffd8140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-22T23:21:40.520 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.519+0000 7f324ffd8140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-22T23:21:40.532 DEBUG:teuthology.orchestra.run.trial189:> set -ex 2026-02-22T23:21:40.533 DEBUG:teuthology.orchestra.run.trial189:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2026-02-22T23:21:40.546 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.545+0000 7f324ffd8140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-22T23:21:40.570 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.569+0000 7f324ffd8140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-22T23:21:40.595 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.594+0000 7f324ffd8140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-22T23:21:40.598 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64 0 2026-02-22T23:21:40.618 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.617+0000 7f324ffd8140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-22T23:21:40.643 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.644+0000 7f73757cf140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-22T23:21:40.700 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.700+0000 7f73757cf140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-22T23:21:40.703 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.701+0000 7f231d7aa640 10 monclient: get_monmap_and_config 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.701+0000 7f231d7aa640 10 monclient: build_initial_monmap 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.701+0000 7f231d7aa640 10 monclient: monmap: 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:40.702681+0000 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:40.702681+0000 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:40.704 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 1 Processor -- start 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 1 -- start start 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient: init 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient: _add_conns ranks=[2,4,5,1,3,0] 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2318073950 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:40.705 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f231811b470 0x7f231811d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient(hunting): picked mon.noname-b con 0x7f231811b470 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f231811dda0 0x7f2318120190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient(hunting): picked mon.noname-c con 0x7f231811dda0 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23181208b0 con 0x7f2318073950 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2318121ad0 con 0x7f231811b470 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2318121c50 con 0x7f231811dda0 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231d7aa640 10 monclient(hunting): authenticate will time out at 978.385380s 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f23177fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f231811b470 0x7f231811d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.702+0000 7f231cfa9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f231811dda0 0x7f2318120190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:40.706 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f231811b470 0x7f231811d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.189:52050/0 (socket says 10.20.193.189:52050) 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 1 -- 10.20.193.189:0/2682391464 learned_addr learned my addr 10.20.193.189:0/2682391464 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 10 monclient(hunting): get_auth_request con 0x7f231811b470 auth_method 0 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f231cfa9640 10 monclient(hunting): get_auth_request con 0x7f231811dda0 auth_method 0 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f231cfa9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f231cfa9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f231cfa9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f23177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:40.707 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f231cfa9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:40.708 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f231cfa9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:40.708 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.703+0000 7f231cfa9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 10 monclient(hunting): handle_auth_done global_id 4115 payload 274 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 1 -- 10.20.193.189:0/2682391464 >> v2:10.20.193.40:3300/0 conn(0x7f231811b470 msgr2=0x7f231811d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 1 --2- 10.20.193.189:0/2682391464 >> v2:10.20.193.40:3300/0 conn(0x7f231811b470 0x7f231811d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 1 -- 10.20.193.189:0/2682391464 >> v1:10.20.193.80:6789/0 conn(0x7f2318073950 legacy=0x7f2318073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 10 monclient: _finish_hunting 0 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 1 monclient: found mon.noname-c 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:40.714 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.704+0000 7f231cfa9640 1 -- 10.20.193.189:0/2682391464 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2318121e00 con 0x7f231811dda0 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.706+0000 7f231cfa9640 1 --2- 10.20.193.189:0/2682391464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f2318120190 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23180745c0 tx=0x7f230c02f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f2ab7271e4907e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 1 -- 10.20.193.189:0/2682391464 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c03c070 con 0x7f231811dda0 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: dump: 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:40.715 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: _finish_auth 0 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 5 monclient: authenticate success, global_id 4115 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 1 -- 10.20.193.189:0/2682391464 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c02fda0 con 0x7f231811dda0 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 1 -- 10.20.193.189:0/2682391464 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c037660 con 0x7f231811dda0 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:40.716 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f2316ffd640 10 monclient: dump: 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:40.717 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 4 set_mon_vals no callback set 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 10 monclient: shutdown 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 -- 10.20.193.189:0/2682391464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 msgr2=0x7f2318120190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 --2- 10.20.193.189:0/2682391464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f2318120190 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23180745c0 tx=0x7f230c02f130 comp rx=0 tx=0).stop 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 -- 10.20.193.189:0/2682391464 shutdown_connections 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 --2- 10.20.193.189:0/2682391464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f2318120190 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 --2- 10.20.193.189:0/2682391464 >> v2:10.20.193.40:3300/0 conn(0x7f231811b470 0x7f231811d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 -- 10.20.193.189:0/2682391464 >> 10.20.193.189:0/2682391464 conn(0x7f231806f210 msgr2=0x7f2318071630 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 -- 10.20.193.189:0/2682391464 shutdown_connections 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.709+0000 7f231d7aa640 1 -- 10.20.193.189:0/2682391464 wait complete. 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient: build_initial_monmap 2026-02-22T23:21:40.718 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient: monmap: 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 librados: starting msgr at 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 librados: starting objecter 2026-02-22T23:21:40.719 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 Processor -- start 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 -- start start 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 librados: setting wanted keys 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 librados: calling monclient init 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient: init 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2318073950 0x7f2318112250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): picked mon.noname-b con 0x7f2318073950 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f231811b470 0x7f2318112790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): picked mon.noname-a con 0x7f231811b470 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f23181b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:40.720 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): picked mon.noname-c con 0x7f231811dda0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2318120f00 con 0x7f231811b470 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2318120d80 con 0x7f2318073950 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2318121080 con 0x7f231811dda0 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231d7aa640 10 monclient(hunting): authenticate will time out at 978.393459s 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231cfa9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f23181b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f2317fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2318073950 0x7f2318112250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231cfa9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f23181b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60154/0 (socket says 10.20.193.189:60154) 2026-02-22T23:21:40.721 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231cfa9640 1 -- 10.20.193.189:0/3930866523 learned_addr learned my addr 10.20.193.189:0/3930866523 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231cfa9640 10 monclient(hunting): get_auth_request con 0x7f231811dda0 auth_method 0 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231cfa9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231cfa9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.710+0000 7f231cfa9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f2317fff640 10 monclient(hunting): get_auth_request con 0x7f2318073950 auth_method 0 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f2317fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f2317fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f2317fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f231811b470 0x7f2318112790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f231cfa9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f231cfa9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f231cfa9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:40.722 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 10 monclient(hunting): get_auth_request con 0x7f231811b470 auth_method 0 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f2317fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f2317fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f2317fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.711+0000 7f23177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 10 monclient(hunting): handle_auth_done global_id 4118 payload 824 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 1 -- 10.20.193.189:0/3930866523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2318073950 msgr2=0x7f2318112250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2318073950 0x7f2318112250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.723 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 1 -- 10.20.193.189:0/3930866523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f231811b470 msgr2=0x7f2318112790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f231811b470 0x7f2318112790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 10 monclient: _finish_hunting 0 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 1 monclient: found mon.noname-c 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.712+0000 7f231cfa9640 1 -- 10.20.193.189:0/3930866523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23181b7cc0 con 0x7f231811dda0 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.713+0000 7f231cfa9640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f23181b7780 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f230c03a040 tx=0x7f230c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.713+0000 7f2314ff9640 10 client.?.objecter ms_handle_connect 0x7f231811dda0 2026-02-22T23:21:40.724 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.713+0000 7f2314ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:40.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f2314ff9640 1 -- 10.20.193.189:0/3930866523 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c03c050 con 0x7f231811dda0 2026-02-22T23:21:40.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f2314ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:40.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f2314ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:40.726 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f2314ff9640 10 monclient: dump: 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f2314ff9640 10 monclient: _finish_auth 0 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f2314ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:21:40.727 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f2314ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 5 monclient: authenticate success, global_id 4118 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 10 monclient: _renew_subs 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23181b7e90 con 0x7f231811dda0 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 10 client.4118.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 10 monclient: _renew_subs 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23181b8240 con 0x7f231811dda0 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.714+0000 7f231d7aa640 1 librados: init done 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f231d7aa640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f231d7aa640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f231d7aa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2318121490 con 0x7f231811dda0 2026-02-22T23:21:40.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f2314ff9640 1 -- 10.20.193.189:0/3930866523 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c048070 con 0x7f231811dda0 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f2314ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f2314ff9640 1 -- 10.20.193.189:0/3930866523 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c041810 con 0x7f231811dda0 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f2314ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f2314ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f2314ff9640 10 monclient: dump: 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:40.729 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:40.730 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.715+0000 7f23167fc640 4 set_mon_vals no callback set 2026-02-22T23:21:40.730 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.723+0000 7f73757cf140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-22T23:21:40.746 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.746+0000 7f73757cf140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-22T23:21:40.786 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.785+0000 7f2314ff9640 1 -- 10.20.193.189:0/3930866523 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f230c041a30 con 0x7f231811dda0 2026-02-22T23:21:40.786 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.785+0000 7f2314ff9640 1 -- 10.20.193.189:0/3930866523 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f230c04daa0 con 0x7f231811dda0 2026-02-22T23:21:40.786 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.785+0000 7f2314ff9640 10 client.4118.objecter ms_dispatch2 0x7f2318000b90 osd_map(2..2 src has 1..2) 2026-02-22T23:21:40.786 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.785+0000 7f2314ff9640 3 client.4118.objecter handle_osd_map got epochs [2,2] > 0 2026-02-22T23:21:40.786 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.785+0000 7f2314ff9640 3 client.4118.objecter handle_osd_map decoding full epoch 2 2026-02-22T23:21:40.786 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.785+0000 7f2314ff9640 20 client.4118.objecter dump_active .. 0 homeless 2026-02-22T23:21:40.789 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.789+0000 7f2314ff9640 1 -- 10.20.193.189:0/3930866523 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f230c04d460 con 0x7f231811dda0 2026-02-22T23:21:40.789 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.789+0000 7f2314ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:40.789 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.789+0000 7f2314ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:40.800 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.801+0000 7f73757cf140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:21:40.810 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.809+0000 7f324ffd8140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-22T23:21:40.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.821+0000 7f231d7aa640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64", "id": 0}] 2026-02-22T23:21:40.822 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.821+0000 7f73757cf140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-22T23:21:40.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.821+0000 7f231d7aa640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64", "id": 0}] 2026-02-22T23:21:40.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.821+0000 7f231d7aa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:40.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.821+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64", "id": 0} v 0) -- 0x7f2318115950 con 0x7f231811dda0 2026-02-22T23:21:40.865 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.864+0000 7f324ffd8140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-22T23:21:40.874 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:40.874+0000 7f73757cf140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:21:40.888 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.887+0000 7f324ffd8140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-22T23:21:40.911 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.910+0000 7f324ffd8140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-22T23:21:40.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f2314ff9640 1 -- 10.20.193.189:0/3930866523 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f230c04d640 con 0x7f231811dda0 2026-02-22T23:21:40.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f2314ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64", "id": 0}] 2026-02-22T23:21:40.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f2314ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:40.917 INFO:teuthology.orchestra.run.trial189.stdout:0 2026-02-22T23:21:40.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 10 librados: watch_flush enter 2026-02-22T23:21:40.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 10 librados: watch_flush exit 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 20 client.4118.objecter shutdown clearing up homeless session... 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 10 client.4118.objecter successfully canceled tick 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 10 monclient: shutdown 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 msgr2=0x7f23181b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f23181b7780 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f230c03a040 tx=0x7f230c004b40 comp rx=0 tx=0).stop 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 shutdown_connections 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f231811dda0 0x7f23181b7780 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f231811b470 0x7f2318112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 --2- 10.20.193.189:0/3930866523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2318073950 0x7f2318112250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:40.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 >> 10.20.193.189:0/3930866523 conn(0x7f231806f210 msgr2=0x7f231811e710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:40.920 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 shutdown_connections 2026-02-22T23:21:40.920 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 -- 10.20.193.189:0/3930866523 wait complete. 2026-02-22T23:21:40.920 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:40.917+0000 7f231d7aa640 1 librados: shutdown 2026-02-22T23:21:40.926 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new d0777c8c-8788-440c-b557-b72b5f0ec2f4 1 2026-02-22T23:21:40.965 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.964+0000 7f324ffd8140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:21:40.986 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:40.984+0000 7f324ffd8140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-22T23:21:41.030 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient: get_monmap_and_config 2026-02-22T23:21:41.030 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.030 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient: monmap: 2026-02-22T23:21:41.030 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.030 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.030 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:41.030899+0000 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:41.030899+0000 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:41.031 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 1 Processor -- start 2026-02-22T23:21:41.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 1 -- start start 2026-02-22T23:21:41.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient: init 2026-02-22T23:21:41.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient: _add_conns ranks=[0,1,4,5,3,2] 2026-02-22T23:21:41.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2d3c11a960 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:41.034 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f2d3c11d670 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f2d3c121210 0x7f2d3c1236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient(hunting): picked mon.noname-b con 0x7f2d3c121210 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3c123df0 con 0x7f2d3c11a960 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3c125010 con 0x7f2d3c11d670 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3c126210 con 0x7f2d3c121210 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.030+0000 7f2d41990640 10 monclient(hunting): authenticate will time out at 978.713703s 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.031+0000 7f2d3b7fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f2d3c121210 0x7f2d3c1236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.031+0000 7f2d3affd640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f2d3c11a960 0x7f2d3c11ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.189:53832/0 (socket says 10.20.193.189:53832) 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.031+0000 7f2d3affd640 1 -- 10.20.193.189:0/1405812415 learned_addr learned my addr 10.20.193.189:0/1405812415 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:41.035 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.031+0000 7f2d3b7fe640 10 monclient(hunting): get_auth_request con 0x7f2d3c121210 auth_method 0 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.031+0000 7f2d3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.031+0000 7f2d3b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.031+0000 7f2d3b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.032+0000 7f2d3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.032+0000 7f2d3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.032+0000 7f2d3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 10 monclient(hunting): handle_auth_done global_id 4120 payload 274 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 1 -- 10.20.193.189:0/1405812415 >> v1:10.20.193.40:6789/0 conn(0x7f2d3c11d670 legacy=0x7f2d3c11fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 1 -- 10.20.193.189:0/1405812415 >> v1:10.20.193.6:6789/0 conn(0x7f2d3c11a960 legacy=0x7f2d3c11ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 1 monclient: found mon.noname-b 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.036 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:41.037 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.033+0000 7f2d3b7fe640 1 -- 10.20.193.189:0/1405812415 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c126390 con 0x7f2d3c121210 2026-02-22T23:21:41.037 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.036+0000 7f2d3b7fe640 1 --2- 10.20.193.189:0/1405812415 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c121210 0x7f2d3c1236a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009d90 tx=0x7f2d2c02f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809658dc1af09309 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.037 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:41.036+0000 7f324ffd8140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:21:41.043 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.038+0000 7f2d39ffb640 1 -- 10.20.193.189:0/1405812415 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c03c070 con 0x7f2d3c121210 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.038+0000 7f2d39ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.038+0000 7f2d39ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.038+0000 7f2d39ffb640 10 monclient: dump: 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.044 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 1 -- 10.20.193.189:0/1405812415 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2c037440 con 0x7f2d3c121210 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 5 monclient: authenticate success, global_id 4120 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:41.045 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 1 -- 10.20.193.189:0/1405812415 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c037810 con 0x7f2d3c121210 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 4 set_mon_vals no callback set 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d39ffb640 10 monclient: dump: 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.046 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient: shutdown 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- 10.20.193.189:0/1405812415 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c121210 msgr2=0x7f2d3c1236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 --2- 10.20.193.189:0/1405812415 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c121210 0x7f2d3c1236a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009d90 tx=0x7f2d2c02f1e0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- 10.20.193.189:0/1405812415 shutdown_connections 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 --2- 10.20.193.189:0/1405812415 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c121210 0x7f2d3c1236a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- 10.20.193.189:0/1405812415 >> 10.20.193.189:0/1405812415 conn(0x7f2d3c06dfb0 msgr2=0x7f2d3c06e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- 10.20.193.189:0/1405812415 shutdown_connections 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- 10.20.193.189:0/1405812415 wait complete. 2026-02-22T23:21:41.047 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient: monmap: 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 librados: starting msgr at 2026-02-22T23:21:41.048 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 librados: starting objecter 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 Processor -- start 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- start start 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 librados: setting wanted keys 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 librados: calling monclient init 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient: init 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c11a960 0x7f2d3c1b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): picked mon.noname-b con 0x7f2d3c11a960 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2d3c11d670 0x7f2d3c1b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): picked mon.noname-c con 0x7f2d3c11d670 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2d3c121210 0x7f2d3c118a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.049 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): picked mon.noname-a con 0x7f2d3c121210 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c125660 con 0x7f2d3c121210 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1254e0 con 0x7f2d3c11a960 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c1257e0 con 0x7f2d3c11d670 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.039+0000 7f2d41990640 10 monclient(hunting): authenticate will time out at 978.722985s 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2d3c11d670 0x7f2d3c1b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c11a960 0x7f2d3c1b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2d3c11d670 0x7f2d3c1b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60168/0 (socket says 10.20.193.189:60168) 2026-02-22T23:21:41.050 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c11a960 0x7f2d3c1b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.189:52078/0 (socket says 10.20.193.189:52078) 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 1 -- 10.20.193.189:0/3141073953 learned_addr learned my addr 10.20.193.189:0/3141073953 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 10 monclient(hunting): get_auth_request con 0x7f2d3c11d670 auth_method 0 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 10 monclient(hunting): get_auth_request con 0x7f2d3c11a960 auth_method 0 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.051 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.040+0000 7f2d3a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.041+0000 7f2d3b7fe640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2d3c121210 0x7f2d3c118a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.041+0000 7f2d3b7fe640 10 monclient(hunting): get_auth_request con 0x7f2d3c121210 auth_method 0 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.041+0000 7f2d3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.041+0000 7f2d3b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.041+0000 7f2d3b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 10 monclient(hunting): handle_auth_done global_id 4126 payload 824 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 1 -- 10.20.193.189:0/3141073953 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2d3c11d670 msgr2=0x7f2d3c1b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2d3c11d670 0x7f2d3c1b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 1 -- 10.20.193.189:0/3141073953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2d3c121210 msgr2=0x7f2d3c118a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.052 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2d3c121210 0x7f2d3c118a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 1 monclient: found mon.noname-b 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.042+0000 7f2d3affd640 1 -- 10.20.193.189:0/3141073953 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c1192d0 con 0x7f2d3c11a960 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.043+0000 7f2d3affd640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c11a960 0x7f2d3c1b1600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000c500 tx=0x7f2d3000c9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.043+0000 7f2d13fff640 10 client.?.objecter ms_handle_connect 0x7f2d3c11a960 2026-02-22T23:21:41.053 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.043+0000 7f2d13fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:41.055 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 1 -- 10.20.193.189:0/3141073953 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30017070 con 0x7f2d3c11a960 2026-02-22T23:21:41.055 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.055 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:41.055 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: dump: 2026-02-22T23:21:41.055 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.055 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.056 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 1 -- 10.20.193.189:0/3141073953 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d30005de0 con 0x7f2d3c11a960 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 1 -- 10.20.193.189:0/3141073953 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d300072b0 con 0x7f2d3c11a960 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d13fff640 10 monclient: dump: 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.057 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d397fa640 4 set_mon_vals no callback set 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 5 monclient: authenticate success, global_id 4126 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 10 monclient: _renew_subs 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c1195c0 con 0x7f2d3c11a960 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 10 client.4126.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 10 monclient: _renew_subs 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c1b8610 con 0x7f2d3c11a960 2026-02-22T23:21:41.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.044+0000 7f2d41990640 1 librados: init done 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.045+0000 7f2d41990640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.045+0000 7f2d41990640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.045+0000 7f2d41990640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.045+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3c125cc0 con 0x7f2d3c11a960 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.046+0000 7f2d13fff640 1 -- 10.20.193.189:0/3141073953 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2d30007450 con 0x7f2d3c11a960 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.046+0000 7f2d13fff640 1 -- 10.20.193.189:0/3141073953 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f2d300127b0 con 0x7f2d3c11a960 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.046+0000 7f2d13fff640 10 client.4126.objecter ms_dispatch2 0x7f2d3c000b90 osd_map(3..3 src has 1..3) 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.046+0000 7f2d13fff640 3 client.4126.objecter handle_osd_map got epochs [3,3] > 0 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.046+0000 7f2d13fff640 3 client.4126.objecter handle_osd_map decoding full epoch 3 2026-02-22T23:21:41.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.046+0000 7f2d13fff640 20 client.4126.objecter dump_active .. 0 homeless 2026-02-22T23:21:41.086 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.085+0000 7f2d13fff640 1 -- 10.20.193.189:0/3141073953 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f2d3001c440 con 0x7f2d3c11a960 2026-02-22T23:21:41.086 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.086+0000 7f2d13fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.086 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.086+0000 7f2d13fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:41.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.118+0000 7f2d41990640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "d0777c8c-8788-440c-b557-b72b5f0ec2f4", "id": 1}] 2026-02-22T23:21:41.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.118+0000 7f2d41990640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "d0777c8c-8788-440c-b557-b72b5f0ec2f4", "id": 1}] 2026-02-22T23:21:41.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.118+0000 7f2d41990640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:41.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.118+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d0777c8c-8788-440c-b557-b72b5f0ec2f4", "id": 1} v 0) -- 0x7f2d3c1b87e0 con 0x7f2d3c11a960 2026-02-22T23:21:41.162 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d13fff640 1 -- 10.20.193.189:0/3141073953 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d0777c8c-8788-440c-b557-b72b5f0ec2f4", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f2d3001c620 con 0x7f2d3c11a960 2026-02-22T23:21:41.162 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d13fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "d0777c8c-8788-440c-b557-b72b5f0ec2f4", "id": 1}] 2026-02-22T23:21:41.162 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d13fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:41.162 INFO:teuthology.orchestra.run.trial189.stdout:1 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 10 librados: watch_flush enter 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 10 librados: watch_flush exit 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 20 client.4126.objecter shutdown clearing up homeless session... 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 10 client.4126.objecter successfully canceled tick 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 10 monclient: shutdown 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c11a960 msgr2=0x7f2d3c1b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c11a960 0x7f2d3c1b1600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000c500 tx=0x7f2d3000c9d0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.163 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 shutdown_connections 2026-02-22T23:21:41.164 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2d3c121210 0x7f2d3c118a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.164 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2d3c11d670 0x7f2d3c1b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.164 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 --2- 10.20.193.189:0/3141073953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2d3c11a960 0x7f2d3c1b1600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.164 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 >> 10.20.193.189:0/3141073953 conn(0x7f2d3c06dfb0 msgr2=0x7f2d3c11fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.164 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 shutdown_connections 2026-02-22T23:21:41.164 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 -- 10.20.193.189:0/3141073953 wait complete. 2026-02-22T23:21:41.164 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.162+0000 7f2d41990640 1 librados: shutdown 2026-02-22T23:21:41.170 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new b0c8c31e-3748-43a4-ae36-aa42fd9a5030 2 2026-02-22T23:21:41.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.235+0000 7f7ed1032640 10 monclient: get_monmap_and_config 2026-02-22T23:21:41.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.235+0000 7f7ed1032640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.235+0000 7f7ed1032640 10 monclient: monmap: 2026-02-22T23:21:41.238 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.238 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.238 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:41.236794+0000 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:41.236794+0000 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 1 Processor -- start 2026-02-22T23:21:41.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 1 -- start start 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient: init 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient: _add_conns ranks=[2,1,3,4,5,0] 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7ecc073930 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7ecc117280 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f7ecc117e20 0x7f7ecc1182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient(hunting): picked mon.noname-a con 0x7f7ecc117e20 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ecc128400 con 0x7f7ecc117280 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ecc1295e0 con 0x7f7ecc073930 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7ecc12a7c0 con 0x7f7ecc117e20 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ed1032640 10 monclient(hunting): authenticate will time out at 978.919477s 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ecad76640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f7ecc073930 0x7f7ecc073d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.189:56906/0 (socket says 10.20.193.189:56906) 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.236+0000 7f7ecad76640 1 -- 10.20.193.189:0/320937844 learned_addr learned my addr 10.20.193.189:0/320937844 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 1 --2- 10.20.193.189:0/320937844 >> v2:10.20.193.6:3300/0 conn(0x7f7ecc117e20 0x7f7ecc1182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 10 monclient(hunting): get_auth_request con 0x7f7ecc117e20 auth_method 0 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.237+0000 7f7ecb577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 1 -- 10.20.193.189:0/320937844 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2214198908 0 0) 0x7f7ecc1295e0 con 0x7f7ecc073930 2026-02-22T23:21:41.241 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): _negotiate 2026-02-22T23:21:41.242 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.242 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.242 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): my global_id is 4124 2026-02-22T23:21:41.242 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 1 -- 10.20.193.189:0/320937844 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eb8003620 con 0x7f7ecc073930 2026-02-22T23:21:41.249 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 1 -- 10.20.193.189:0/320937844 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132965800 0 0) 0x7f7ecc128400 con 0x7f7ecc117280 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): _negotiate 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 10 monclient(hunting): my global_id is 4129 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ec9d74640 1 -- 10.20.193.189:0/320937844 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ecc1295e0 con 0x7f7ecc117280 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 10 monclient(hunting): handle_auth_done global_id 4110 payload 274 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 1 -- 10.20.193.189:0/320937844 >> v1:10.20.193.80:6789/0 conn(0x7f7ecc073930 legacy=0x7f7ecc073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 1 -- 10.20.193.189:0/320937844 >> v1:10.20.193.40:6789/0 conn(0x7f7ecc117280 legacy=0x7f7ecc117710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 1 monclient: found mon.noname-a 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.238+0000 7f7ecb577640 1 -- 10.20.193.189:0/320937844 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc12a940 con 0x7f7ecc117e20 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.239+0000 7f7ecb577640 1 --2- 10.20.193.189:0/320937844 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1182d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009d90 tx=0x7f7ebc02f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9fb0c1b39cbc11 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 1 -- 10.20.193.189:0/320937844 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc03c070 con 0x7f7ecc117e20 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: dump: 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.251 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 1 -- 10.20.193.189:0/320937844 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc037440 con 0x7f7ecc117e20 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 1 -- 10.20.193.189:0/320937844 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc037810 con 0x7f7ecc117e20 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.252 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ec9d74640 10 monclient: dump: 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 5 monclient: authenticate success, global_id 4110 2026-02-22T23:21:41.253 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 4 set_mon_vals no callback set 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 10 monclient: shutdown 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 1 -- 10.20.193.189:0/320937844 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 msgr2=0x7f7ecc1182d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 1 --2- 10.20.193.189:0/320937844 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1182d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009d90 tx=0x7f7ebc02f1e0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 1 -- 10.20.193.189:0/320937844 shutdown_connections 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 1 --2- 10.20.193.189:0/320937844 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1182d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 1 -- 10.20.193.189:0/320937844 >> 10.20.193.189:0/320937844 conn(0x7f7ecc06f640 msgr2=0x7f7ecc071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 1 -- 10.20.193.189:0/320937844 shutdown_connections 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.242+0000 7f7ed1032640 1 -- 10.20.193.189:0/320937844 wait complete. 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient: monmap: 2026-02-22T23:21:41.254 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 librados: starting msgr at 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 librados: starting objecter 2026-02-22T23:21:41.255 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 Processor -- start 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 -- start start 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 librados: setting wanted keys 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 librados: calling monclient init 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient: init 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ecc073930 0x7f7ecc126900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): picked mon.noname-c con 0x7f7ecc073930 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ecc117280 0x7f7ecc126f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): picked mon.noname-b con 0x7f7ecc117280 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): picked mon.noname-a con 0x7f7ecc117e20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc128a50 con 0x7f7ecc117e20 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1288d0 con 0x7f7ecc117280 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc128bd0 con 0x7f7ecc073930 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ed1032640 10 monclient(hunting): authenticate will time out at 978.926293s 2026-02-22T23:21:41.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecad76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ecc073930 0x7f7ecc126900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7eca575640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ecc117280 0x7f7ecc126f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecb577640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecad76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ecc073930 0x7f7ecc126900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60174/0 (socket says 10.20.193.189:60174) 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecad76640 1 -- 10.20.193.189:0/2579611808 learned_addr learned my addr 10.20.193.189:0/2579611808 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecad76640 10 monclient(hunting): get_auth_request con 0x7f7ecc073930 auth_method 0 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.243+0000 7f7ecad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7eca575640 10 monclient(hunting): get_auth_request con 0x7f7ecc117280 auth_method 0 2026-02-22T23:21:41.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7eca575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7eca575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7eca575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecb577640 10 monclient(hunting): get_auth_request con 0x7f7ecc117e20 auth_method 0 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecb577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecb577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecb577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecb577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecb577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.259 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7ecb577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7eca575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7eca575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.244+0000 7f7eca575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 10 monclient(hunting): handle_auth_done global_id 4113 payload 824 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 1 -- 10.20.193.189:0/2579611808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ecc073930 msgr2=0x7f7ecc126900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ecc073930 0x7f7ecc126900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 1 -- 10.20.193.189:0/2579611808 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ecc117280 msgr2=0x7f7ecc126f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ecc117280 0x7f7ecc126f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 1 monclient: found mon.noname-a 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.260 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecb577640 1 -- 10.20.193.189:0/2579611808 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc1b6060 con 0x7f7ecc117e20 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.245+0000 7f7ecad76640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ecc073930 0x7f7ecc126900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.246+0000 7f7ecb577640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1b57a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc036000 tx=0x7f7ebc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.246+0000 7f7eab7fe640 10 client.?.objecter ms_handle_connect 0x7f7ecc117e20 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.246+0000 7f7eab7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 1 -- 10.20.193.189:0/2579611808 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc03c050 con 0x7f7ecc117e20 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: dump: 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.261 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.262 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 1 -- 10.20.193.189:0/2579611808 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc03d040 con 0x7f7ecc117e20 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 1 -- 10.20.193.189:0/2579611808 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc041810 con 0x7f7ecc117e20 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ec9573640 4 set_mon_vals no callback set 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7eab7fe640 10 monclient: dump: 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.263 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 5 monclient: authenticate success, global_id 4113 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 monclient: _renew_subs 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc1b6230 con 0x7f7ecc117e20 2026-02-22T23:21:41.264 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 client.4113.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 monclient: _renew_subs 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1bc850 con 0x7f7ecc117e20 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 1 librados: init done 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:41.265 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.248+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc118350 con 0x7f7ecc117e20 2026-02-22T23:21:41.266 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.250+0000 7f7eab7fe640 1 -- 10.20.193.189:0/2579611808 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f7ebc0419b0 con 0x7f7ecc117e20 2026-02-22T23:21:41.266 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.250+0000 7f7eab7fe640 1 -- 10.20.193.189:0/2579611808 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7ebc04c830 con 0x7f7ecc117e20 2026-02-22T23:21:41.266 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.250+0000 7f7eab7fe640 10 client.4113.objecter ms_dispatch2 0x7f7ecc000b90 osd_map(4..4 src has 1..4) 2026-02-22T23:21:41.266 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.250+0000 7f7eab7fe640 3 client.4113.objecter handle_osd_map got epochs [4,4] > 0 2026-02-22T23:21:41.266 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.250+0000 7f7eab7fe640 3 client.4113.objecter handle_osd_map decoding full epoch 4 2026-02-22T23:21:41.266 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.250+0000 7f7eab7fe640 20 client.4113.objecter dump_active .. 0 homeless 2026-02-22T23:21:41.268 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:41.268+0000 7f73757cf140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-22T23:21:41.290 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:41.290+0000 7f73757cf140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-22T23:21:41.299 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.298+0000 7f7eab7fe640 1 -- 10.20.193.189:0/2579611808 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f7ebc041dc0 con 0x7f7ecc117e20 2026-02-22T23:21:41.299 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.298+0000 7f7eab7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.299 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.298+0000 7f7eab7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:41.331 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.331+0000 7f7ed1032640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "b0c8c31e-3748-43a4-ae36-aa42fd9a5030", "id": 2}] 2026-02-22T23:21:41.331 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.331+0000 7f7ed1032640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "b0c8c31e-3748-43a4-ae36-aa42fd9a5030", "id": 2}] 2026-02-22T23:21:41.331 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.331+0000 7f7ed1032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:41.331 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.331+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b0c8c31e-3748-43a4-ae36-aa42fd9a5030", "id": 2} v 0) -- 0x7f7ecc1bca20 con 0x7f7ecc117e20 2026-02-22T23:21:41.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7eab7fe640 1 -- 10.20.193.189:0/2579611808 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b0c8c31e-3748-43a4-ae36-aa42fd9a5030", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7ebc04c410 con 0x7f7ecc117e20 2026-02-22T23:21:41.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7eab7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "b0c8c31e-3748-43a4-ae36-aa42fd9a5030", "id": 2}] 2026-02-22T23:21:41.351 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7eab7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:41.352 INFO:teuthology.orchestra.run.trial189.stdout:2 2026-02-22T23:21:41.353 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 10 librados: watch_flush enter 2026-02-22T23:21:41.353 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 10 librados: watch_flush exit 2026-02-22T23:21:41.353 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 20 client.4113.objecter shutdown clearing up homeless session... 2026-02-22T23:21:41.353 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 10 client.4113.objecter successfully canceled tick 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 10 monclient: shutdown 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 msgr2=0x7f7ecc1b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.351+0000 7f7ed1032640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1b57a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc036000 tx=0x7f7ebc004b40 comp rx=0 tx=0).stop 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 shutdown_connections 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ecc117e20 0x7f7ecc1b57a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ecc117280 0x7f7ecc126f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 --2- 10.20.193.189:0/2579611808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ecc073930 0x7f7ecc126900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 >> 10.20.193.189:0/2579611808 conn(0x7f7ecc06f640 msgr2=0x7f7ecc070950 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 shutdown_connections 2026-02-22T23:21:41.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 -- 10.20.193.189:0/2579611808 wait complete. 2026-02-22T23:21:41.355 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.352+0000 7f7ed1032640 1 librados: shutdown 2026-02-22T23:21:41.359 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 542e6008-d73b-45c7-a6d1-f34712ec1541 3 2026-02-22T23:21:41.425 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.422+0000 7f400833e640 10 monclient: get_monmap_and_config 2026-02-22T23:21:41.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.422+0000 7f400833e640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.426 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.422+0000 7f400833e640 10 monclient: monmap: 2026-02-22T23:21:41.426 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.426 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.426 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:41.423854+0000 2026-02-22T23:21:41.426 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:41.423854+0000 2026-02-22T23:21:41.426 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 1 Processor -- start 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 1 -- start start 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient: init 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.427 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient: _add_conns ranks=[4,3,0,2,1,5] 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f40001175d0 0x7f40001179b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient(hunting): picked mon.noname-b con 0x7f40001175d0 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f4000112180 0x7f4000112630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient(hunting): picked mon.noname-a con 0x7f4000112180 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4000112b70 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f400011fa30 con 0x7f4000112b70 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4000118130 con 0x7f4000112180 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4000120c10 con 0x7f40001175d0 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.428 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f400833e640 10 monclient(hunting): authenticate will time out at 979.106506s 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f40058b2640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f4000112180 0x7f4000112630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.423+0000 7f40060b3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f40001175d0 0x7f40001179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f40001175d0 0x7f40001179b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.189:52092/0 (socket says 10.20.193.189:52092) 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 1 -- 10.20.193.189:0/3853614674 learned_addr learned my addr 10.20.193.189:0/3853614674 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 10 monclient(hunting): get_auth_request con 0x7f40001175d0 auth_method 0 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40058b2640 10 monclient(hunting): get_auth_request con 0x7f4000112180 auth_method 0 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40058b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40058b2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.429 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40058b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40060b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40058b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40058b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.424+0000 7f40058b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 10 monclient(hunting): handle_auth_done global_id 4135 payload 274 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 1 -- 10.20.193.189:0/3853614674 >> v2:10.20.193.6:3300/0 conn(0x7f4000112180 msgr2=0x7f4000112630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 1 --2- 10.20.193.189:0/3853614674 >> v2:10.20.193.6:3300/0 conn(0x7f4000112180 0x7f4000112630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 1 -- 10.20.193.189:0/3853614674 >> v1:10.20.193.6:6789/0 conn(0x7f4000112b70 legacy=0x7f400011e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 1 monclient: found mon.noname-b 2026-02-22T23:21:41.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:41.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.425+0000 7f40060b3640 1 -- 10.20.193.189:0/3853614674 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4000120d90 con 0x7f40001175d0 2026-02-22T23:21:41.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.426+0000 7f40060b3640 1 --2- 10.20.193.189:0/3853614674 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 0x7f40001179b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4000073760 tx=0x7f3ff002ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b291acc9a989d31a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 1 -- 10.20.193.189:0/3853614674 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff003c070 con 0x7f40001175d0 2026-02-22T23:21:41.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:41.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: dump: 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 1 -- 10.20.193.189:0/3853614674 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff002fa90 con 0x7f40001175d0 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f400833e640 5 monclient: authenticate success, global_id 4135 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 1 -- 10.20.193.189:0/3853614674 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff002fe60 con 0x7f40001175d0 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f40050b1640 10 monclient: dump: 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.440 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f400833e640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.427+0000 7f400833e640 4 set_mon_vals no callback set 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 10 monclient: shutdown 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 -- 10.20.193.189:0/3853614674 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 msgr2=0x7f40001179b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 --2- 10.20.193.189:0/3853614674 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 0x7f40001179b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4000073760 tx=0x7f3ff002ee60 comp rx=0 tx=0).stop 2026-02-22T23:21:41.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 -- 10.20.193.189:0/3853614674 shutdown_connections 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 --2- 10.20.193.189:0/3853614674 >> v2:10.20.193.6:3300/0 conn(0x7f4000112180 0x7f4000112630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 --2- 10.20.193.189:0/3853614674 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 0x7f40001179b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 -- 10.20.193.189:0/3853614674 >> 10.20.193.189:0/3853614674 conn(0x7f400006f640 msgr2=0x7f4000071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 -- 10.20.193.189:0/3853614674 shutdown_connections 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 -- 10.20.193.189:0/3853614674 wait complete. 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 10 monclient: monmap: 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:41.442 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 librados: starting msgr at 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 librados: starting objecter 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 Processor -- start 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 -- start start 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 librados: setting wanted keys 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 1 librados: calling monclient init 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.428+0000 7f400833e640 10 monclient: init 2026-02-22T23:21:41.443 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4000112180 0x7f40001a7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): picked mon.noname-c con 0x7f4000112180 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4000112b70 0x7f40001a8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): picked mon.noname-a con 0x7f4000112b70 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 0x7f40001acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): picked mon.noname-b con 0x7f40001175d0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f40001200c0 con 0x7f4000112b70 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f400011ff40 con 0x7f40001175d0 2026-02-22T23:21:41.444 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4000120240 con 0x7f4000112180 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f400833e640 10 monclient(hunting): authenticate will time out at 979.112074s 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4000112180 0x7f40001a7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4000112b70 0x7f40001a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40068b4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 0x7f40001acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4000112180 0x7f40001a7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60176/0 (socket says 10.20.193.189:60176) 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4000112b70 0x7f40001a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.189:36794/0 (socket says 10.20.193.189:36794) 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 1 -- 10.20.193.189:0/110851802 learned_addr learned my addr 10.20.193.189:0/110851802 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 10 monclient(hunting): get_auth_request con 0x7f4000112180 auth_method 0 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 10 monclient(hunting): get_auth_request con 0x7f4000112b70 auth_method 0 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40068b4640 10 monclient(hunting): get_auth_request con 0x7f40001175d0 auth_method 0 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40068b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40068b4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40068b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40058b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.429+0000 7f40060b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.430+0000 7f40068b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.430+0000 7f40068b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.430+0000 7f40068b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 10 monclient(hunting): handle_auth_done global_id 4130 payload 824 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 1 -- 10.20.193.189:0/110851802 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 msgr2=0x7f40001acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 1 --2- 10.20.193.189:0/110851802 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 0x7f40001acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 1 -- 10.20.193.189:0/110851802 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4000112b70 msgr2=0x7f40001a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 1 --2- 10.20.193.189:0/110851802 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4000112b70 0x7f40001a8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.447 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 1 monclient: found mon.noname-c 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.431+0000 7f40060b3640 1 -- 10.20.193.189:0/110851802 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40001ad550 con 0x7f4000112180 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.432+0000 7f40060b3640 1 --2- 10.20.193.189:0/110851802 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4000112180 0x7f40001a7f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009a70 tx=0x7f3ff00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.432+0000 7f3fe6ffd640 10 client.?.objecter ms_handle_connect 0x7f4000112180 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.432+0000 7f3fe6ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 1 -- 10.20.193.189:0/110851802 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff003c040 con 0x7f4000112180 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: dump: 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.448 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 5 monclient: authenticate success, global_id 4130 2026-02-22T23:21:41.449 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 monclient: _renew_subs 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40001ad780 con 0x7f4000112180 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 client.4130.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 monclient: _renew_subs 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40001b3e00 con 0x7f4000112180 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 1 librados: init done 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 1 -- 10.20.193.189:0/110851802 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff002f990 con 0x7f4000112180 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 1 -- 10.20.193.189:0/110851802 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0042830 con 0x7f4000112180 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:41.450 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f40048b0640 4 set_mon_vals no callback set 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f3fe6ffd640 10 monclient: dump: 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.451 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.433+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4000112630 con 0x7f4000112180 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.435+0000 7f3fe6ffd640 1 -- 10.20.193.189:0/110851802 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3ff00429d0 con 0x7f4000112180 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.435+0000 7f3fe6ffd640 1 -- 10.20.193.189:0/110851802 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f3ff004b400 con 0x7f4000112180 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.435+0000 7f3fe6ffd640 10 client.4130.objecter ms_dispatch2 0x7f4000000b90 osd_map(5..5 src has 1..5) 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.435+0000 7f3fe6ffd640 3 client.4130.objecter handle_osd_map got epochs [5,5] > 0 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.435+0000 7f3fe6ffd640 3 client.4130.objecter handle_osd_map decoding full epoch 5 2026-02-22T23:21:41.452 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.435+0000 7f3fe6ffd640 20 client.4130.objecter dump_active .. 0 homeless 2026-02-22T23:21:41.453 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:41.430+0000 7f324ffd8140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-22T23:21:41.454 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:41.453+0000 7f324ffd8140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-22T23:21:41.474 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.474+0000 7f3fe6ffd640 1 -- 10.20.193.189:0/110851802 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f3ff0042cf0 con 0x7f4000112180 2026-02-22T23:21:41.475 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.474+0000 7f3fe6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.475 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.474+0000 7f3fe6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:41.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.507+0000 7f400833e640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "542e6008-d73b-45c7-a6d1-f34712ec1541", "id": 3}] 2026-02-22T23:21:41.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.507+0000 7f400833e640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "542e6008-d73b-45c7-a6d1-f34712ec1541", "id": 3}] 2026-02-22T23:21:41.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.507+0000 7f400833e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.507+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "542e6008-d73b-45c7-a6d1-f34712ec1541", "id": 3} v 0) -- 0x7f40001b4030 con 0x7f4000112180 2026-02-22T23:21:41.539 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.538+0000 7f3fe6ffd640 1 -- 10.20.193.189:0/110851802 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "542e6008-d73b-45c7-a6d1-f34712ec1541", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3ff0055b60 con 0x7f4000112180 2026-02-22T23:21:41.539 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.538+0000 7f3fe6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "542e6008-d73b-45c7-a6d1-f34712ec1541", "id": 3}] 2026-02-22T23:21:41.539 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.538+0000 7f3fe6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:41.539 INFO:teuthology.orchestra.run.trial189.stdout:3 2026-02-22T23:21:41.541 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 10 librados: watch_flush enter 2026-02-22T23:21:41.541 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 10 librados: watch_flush exit 2026-02-22T23:21:41.541 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 20 client.4130.objecter shutdown clearing up homeless session... 2026-02-22T23:21:41.541 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 10 client.4130.objecter successfully canceled tick 2026-02-22T23:21:41.541 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 10 monclient: shutdown 2026-02-22T23:21:41.541 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.541 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4000112180 msgr2=0x7f40001a7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 --2- 10.20.193.189:0/110851802 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4000112180 0x7f40001a7f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009a70 tx=0x7f3ff00047c0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 shutdown_connections 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 --2- 10.20.193.189:0/110851802 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f40001175d0 0x7f40001acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 --2- 10.20.193.189:0/110851802 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4000112b70 0x7f40001a8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 --2- 10.20.193.189:0/110851802 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4000112180 0x7f40001a7f00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 >> 10.20.193.189:0/110851802 conn(0x7f400006f640 msgr2=0x7f40000714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 shutdown_connections 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 -- 10.20.193.189:0/110851802 wait complete. 2026-02-22T23:21:41.542 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.539+0000 7f400833e640 1 librados: shutdown 2026-02-22T23:21:41.547 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 075bb353-79d1-4bb1-b765-5b243bde4a2d 4 2026-02-22T23:21:41.614 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient: get_monmap_and_config 2026-02-22T23:21:41.614 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.614 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient: monmap: 2026-02-22T23:21:41.614 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.614 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.614 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:41.612293+0000 2026-02-22T23:21:41.614 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:41.612293+0000 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 1 Processor -- start 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 1 -- start start 2026-02-22T23:21:41.615 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient: init 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient: _add_conns ranks=[3,0,5,4,2,1] 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7feec8073930 0x7feec8073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient(hunting): picked mon.noname-a con 0x7feec8073930 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7feec8074280 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7feec8118410 0x7feec8116930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient(hunting): picked mon.noname-c con 0x7feec8118410 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feec8127360 con 0x7feec8074280 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.616 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7feec8117050 con 0x7feec8073930 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7feec8128540 con 0x7feec8118410 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.611+0000 7feece786640 10 monclient(hunting): authenticate will time out at 979.294937s 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7feec8118410 0x7feec8116930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feec7fff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7feec8073930 0x7feec8073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7feec8118410 0x7feec8116930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60186/0 (socket says 10.20.193.189:60186) 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 1 -- 10.20.193.189:0/3371070253 learned_addr learned my addr 10.20.193.189:0/3371070253 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 10 monclient(hunting): get_auth_request con 0x7feec8118410 auth_method 0 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.617 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feec7fff640 10 monclient(hunting): get_auth_request con 0x7feec8073930 auth_method 0 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feec7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feec7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feec7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.612+0000 7feecccfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 10 monclient(hunting): handle_auth_done global_id 4133 payload 274 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 1 -- 10.20.193.189:0/3371070253 >> v2:10.20.193.6:3300/0 conn(0x7feec8073930 msgr2=0x7feec8073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 1 --2- 10.20.193.189:0/3371070253 >> v2:10.20.193.6:3300/0 conn(0x7feec8073930 0x7feec8073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 1 -- 10.20.193.189:0/3371070253 >> v1:10.20.193.6:6789/0 conn(0x7feec8074280 legacy=0x7feec8116200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 1 monclient: found mon.noname-c 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.618 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.619 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.613+0000 7feecccfc640 1 -- 10.20.193.189:0/3371070253 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec81286f0 con 0x7feec8118410 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.614+0000 7feecccfc640 1 --2- 10.20.193.189:0/3371070253 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8118410 0x7feec8116930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feec81183e0 tx=0x7feebc02f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afee94def127e71d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 1 -- 10.20.193.189:0/3371070253 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc02fcf0 con 0x7feec8118410 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: dump: 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.623 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 1 -- 10.20.193.189:0/3371070253 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc0373f0 con 0x7feec8118410 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 5 monclient: authenticate success, global_id 4133 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 1 -- 10.20.193.189:0/3371070253 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0377c0 con 0x7feec8118410 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 4 set_mon_vals no callback set 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.624 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feec6ffd640 10 monclient: dump: 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 10 monclient: shutdown 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 -- 10.20.193.189:0/3371070253 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8118410 msgr2=0x7feec8116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 --2- 10.20.193.189:0/3371070253 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8118410 0x7feec8116930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feec81183e0 tx=0x7feebc02f230 comp rx=0 tx=0).stop 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 -- 10.20.193.189:0/3371070253 shutdown_connections 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 --2- 10.20.193.189:0/3371070253 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8118410 0x7feec8116930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.625 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 --2- 10.20.193.189:0/3371070253 >> v2:10.20.193.6:3300/0 conn(0x7feec8073930 0x7feec8073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 -- 10.20.193.189:0/3371070253 >> 10.20.193.189:0/3371070253 conn(0x7feec806f640 msgr2=0x7feec8071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 -- 10.20.193.189:0/3371070253 shutdown_connections 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 1 -- 10.20.193.189:0/3371070253 wait complete. 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.616+0000 7feece786640 10 monclient: monmap: 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 librados: starting msgr at 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 librados: starting objecter 2026-02-22T23:21:41.626 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 Processor -- start 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 -- start start 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 librados: setting wanted keys 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 librados: calling monclient init 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient: init 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feec8073930 0x7feec81afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): picked mon.noname-a con 0x7feec8073930 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8074280 0x7feec81b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): picked mon.noname-c con 0x7feec8074280 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feec8118410 0x7feec81b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): picked mon.noname-b con 0x7feec8118410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7feec8127990 con 0x7feec8073930 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7feec8127810 con 0x7feec8118410 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7feec8127b10 con 0x7feec8074280 2026-02-22T23:21:41.627 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feece786640 10 monclient(hunting): authenticate will time out at 979.300359s 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8074280 0x7feec81b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feecccfc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feec8118410 0x7feec81b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8074280 0x7feec81b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60202/0 (socket says 10.20.193.189:60202) 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 1 -- 10.20.193.189:0/2419194549 learned_addr learned my addr 10.20.193.189:0/2419194549 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 10 monclient(hunting): get_auth_request con 0x7feec8074280 auth_method 0 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec7fff640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feec8073930 0x7feec81afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.617+0000 7feec77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feecccfc640 10 monclient(hunting): get_auth_request con 0x7feec8118410 auth_method 0 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feecccfc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feecccfc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feecccfc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.628 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec7fff640 10 monclient(hunting): get_auth_request con 0x7feec8073930 auth_method 0 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feec7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feecccfc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feecccfc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.618+0000 7feecccfc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 10 monclient(hunting): handle_auth_done global_id 4136 payload 824 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 1 -- 10.20.193.189:0/2419194549 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feec8118410 msgr2=0x7feec81b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feec8118410 0x7feec81b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 1 -- 10.20.193.189:0/2419194549 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feec8073930 msgr2=0x7feec81afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feec8073930 0x7feec81afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 1 monclient: found mon.noname-c 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.629 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.619+0000 7feec77fe640 1 -- 10.20.193.189:0/2419194549 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec81b4c90 con 0x7feec8074280 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.620+0000 7feec77fe640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8074280 0x7feec81b02a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c580 tx=0x7feeb800ca50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.620+0000 7feec4ff9640 10 client.?.objecter ms_handle_connect 0x7feec8074280 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.620+0000 7feec4ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8017070 con 0x7feec8074280 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: dump: 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.630 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 5 monclient: authenticate success, global_id 4136 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 10 monclient: _renew_subs 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec81b4f80 con 0x7feec8074280 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 10 client.4136.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 10 monclient: _renew_subs 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec81bc8a0 con 0x7feec8074280 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 1 librados: init done 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb80052d0 con 0x7feec8074280 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80036b0 con 0x7feec8074280 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec4ff9640 10 monclient: dump: 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.631 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feec67fc640 4 set_mon_vals no callback set 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.621+0000 7feece786640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.622+0000 7feece786640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.622+0000 7feece786640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.632 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.622+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec8073d10 con 0x7feec8074280 2026-02-22T23:21:41.633 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.623+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7feeb8003850 con 0x7feec8074280 2026-02-22T23:21:41.633 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.623+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7feeb801c950 con 0x7feec8074280 2026-02-22T23:21:41.633 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.623+0000 7feec4ff9640 10 client.4136.objecter ms_dispatch2 0x7feec8000b90 osd_map(6..6 src has 1..6) 2026-02-22T23:21:41.633 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.623+0000 7feec4ff9640 3 client.4136.objecter handle_osd_map got epochs [6,6] > 0 2026-02-22T23:21:41.633 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.623+0000 7feec4ff9640 3 client.4136.objecter handle_osd_map decoding full epoch 6 2026-02-22T23:21:41.633 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.623+0000 7feec4ff9640 20 client.4136.objecter dump_active .. 0 homeless 2026-02-22T23:21:41.634 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:41.634+0000 7f73757cf140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-22T23:21:41.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.661+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7feeb8022020 con 0x7feec8074280 2026-02-22T23:21:41.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.661+0000 7feec4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.661+0000 7feec4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:41.692 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.692+0000 7feece786640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "075bb353-79d1-4bb1-b765-5b243bde4a2d", "id": 4}] 2026-02-22T23:21:41.693 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.692+0000 7feece786640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "075bb353-79d1-4bb1-b765-5b243bde4a2d", "id": 4}] 2026-02-22T23:21:41.693 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.692+0000 7feece786640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.693 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.692+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "075bb353-79d1-4bb1-b765-5b243bde4a2d", "id": 4} v 0) -- 0x7feec81bcb60 con 0x7feec8074280 2026-02-22T23:21:41.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.751+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7feeb8003b70 con 0x7feec8074280 2026-02-22T23:21:41.794 INFO:tasks.ceph.mgr.y.trial006.stderr:2026-02-22T23:21:41.793+0000 7f324ffd8140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-22T23:21:41.860 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feec4ff9640 1 -- 10.20.193.189:0/2419194549 <== mon.2 v2:10.20.193.80:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "075bb353-79d1-4bb1-b765-5b243bde4a2d", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7feeb801be20 con 0x7feec8074280 2026-02-22T23:21:41.860 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feec4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "075bb353-79d1-4bb1-b765-5b243bde4a2d", "id": 4}] 2026-02-22T23:21:41.860 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feec4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:41.860 INFO:teuthology.orchestra.run.trial189.stdout:4 2026-02-22T23:21:41.860 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 10 librados: watch_flush enter 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 10 librados: watch_flush exit 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 20 client.4136.objecter shutdown clearing up homeless session... 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 10 client.4136.objecter successfully canceled tick 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 10 monclient: shutdown 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8074280 msgr2=0x7feec81b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8074280 0x7feec81b02a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c580 tx=0x7feeb800ca50 comp rx=0 tx=0).stop 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 shutdown_connections 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feec8118410 0x7feec81b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feec8074280 0x7feec81b02a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 --2- 10.20.193.189:0/2419194549 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feec8073930 0x7feec81afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 >> 10.20.193.189:0/2419194549 conn(0x7feec806f640 msgr2=0x7feec8070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 shutdown_connections 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 -- 10.20.193.189:0/2419194549 wait complete. 2026-02-22T23:21:41.861 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.860+0000 7feece786640 1 librados: shutdown 2026-02-22T23:21:41.870 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 52c5895d-8795-4e32-8b07-c4f246dc0503 5 2026-02-22T23:21:41.935 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.933+0000 7fe5c1a58640 10 monclient: get_monmap_and_config 2026-02-22T23:21:41.935 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.933+0000 7fe5c1a58640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.935 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.933+0000 7fe5c1a58640 10 monclient: monmap: 2026-02-22T23:21:41.935 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.935 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.935 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:41.934674+0000 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:41.934674+0000 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 1 Processor -- start 2026-02-22T23:21:41.936 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 1 -- start start 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient: init 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient: _add_conns ranks=[2,0,3,5,1,4] 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe5bc073b00 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe5bc11b4e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fe5bc11eff0 0x7fe5bc121490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient(hunting): picked mon.noname-a con 0x7fe5bc11eff0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5bc121be0 con 0x7fe5bc11b4e0 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5bc122e00 con 0x7fe5bc073b00 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.937 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe5bc124000 con 0x7fe5bc11eff0 2026-02-22T23:21:41.938 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.938 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5c1a58640 10 monclient(hunting): authenticate will time out at 979.617378s 2026-02-22T23:21:41.938 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5baffd640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fe5bc073b00 0x7fe5bc073ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.189:56916/0 (socket says 10.20.193.189:56916) 2026-02-22T23:21:41.938 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.934+0000 7fe5baffd640 1 -- 10.20.193.189:0/52065246 learned_addr learned my addr 10.20.193.189:0/52065246 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:41.945 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.936+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001204718 0 0) 0x7fe5bc122e00 con 0x7fe5bc073b00 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.936+0000 7fe5b9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.936+0000 7fe5b9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.936+0000 7fe5b9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.936+0000 7fe5b9ffb640 10 monclient(hunting): my global_id is 4148 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.936+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5a8003620 con 0x7fe5bc073b00 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4147004610 0 0) 0x7fe5a8003620 con 0x7fe5bc073b00 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 >> v2:10.20.193.6:3300/0 conn(0x7fe5bc11eff0 msgr2=0x7fe5bc121490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 --2- 10.20.193.189:0/52065246 >> v2:10.20.193.6:3300/0 conn(0x7fe5bc11eff0 0x7fe5bc121490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.946 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 >> v1:10.20.193.6:6789/0 conn(0x7fe5bc11b4e0 legacy=0x7fe5bc11d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc124180 con 0x7fe5bc073b00 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fe5b0002f00 con 0x7fe5bc073b00 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:21:41.947 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: dump: 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:21:41.948 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 >> v1:10.20.193.80:6789/0 conn(0x7fe5bc073b00 legacy=0x7fe5bc073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe5a8003ba0 0x7fe5a8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): picked mon.a con 0x7fe5a8003ba0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5a8004900 0x7fe5a8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): picked mon.c con 0x7fe5a8004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8009ab0 0x7fe5a800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): picked mon.b con 0x7fe5a8009ab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc124000 con 0x7fe5a8003ba0 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a800c4f0 con 0x7fe5a8009ab0 2026-02-22T23:21:41.949 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a800c6f0 con 0x7fe5a8004900 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.937+0000 7fe5b9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.938+0000 7fe5ba7fc640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5a8004900 0x7fe5a8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.939+0000 7fe5ba7fc640 10 monclient(hunting): get_auth_request con 0x7fe5a8004900 auth_method 0 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.939+0000 7fe5ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.939+0000 7fe5ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.939+0000 7fe5ba7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.939+0000 7fe5ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.939+0000 7fe5ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.939+0000 7fe5ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 10 monclient(hunting): handle_auth_done global_id 4148 payload 293 2026-02-22T23:21:41.950 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 1 -- 10.20.193.189:0/52065246 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8009ab0 msgr2=0x7fe5a800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8009ab0 0x7fe5a800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 1 -- 10.20.193.189:0/52065246 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe5a8003ba0 msgr2=0x7fe5a8003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe5a8003ba0 0x7fe5a8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 1 monclient: found mon.c 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5bb7fe640 1 -- 10.20.193.189:0/52065246 reap_dead start 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 1 -- 10.20.193.189:0/52065246 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a8003620 con 0x7fe5a8004900 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.951 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5c1a58640 5 monclient: authenticate success, global_id 4148 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5c1a58640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.940+0000 7fe5ba7fc640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5a8004900 0x7fe5a8004d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4002f60 tx=0x7fe5a402eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83858205562276e5 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4002b30 con 0x7fe5a8004900 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 10 monclient: dump: 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.952 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a4005290 con 0x7fe5a8004900 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 1 -- 10.20.193.189:0/52065246 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4003710 con 0x7fe5a8004900 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.953 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 4 set_mon_vals no callback set 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5b9ffb640 10 monclient: dump: 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.954 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 10 monclient: shutdown 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 1 -- 10.20.193.189:0/52065246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5a8004900 msgr2=0x7fe5a8004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5a8004900 0x7fe5a8004d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4002f60 tx=0x7fe5a402eca0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 1 -- 10.20.193.189:0/52065246 shutdown_connections 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 1 --2- 10.20.193.189:0/52065246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5a8004900 0x7fe5a8004d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 1 -- 10.20.193.189:0/52065246 >> 10.20.193.189:0/52065246 conn(0x7fe5bc06f640 msgr2=0x7fe5bc071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 1 -- 10.20.193.189:0/52065246 shutdown_connections 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.942+0000 7fe5c1a58640 1 -- 10.20.193.189:0/52065246 wait complete. 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient: build_initial_monmap 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient: monmap: 2026-02-22T23:21:41.955 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:41.956 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:41.957 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.957 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 librados: starting msgr at 2026-02-22T23:21:41.957 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 librados: starting objecter 2026-02-22T23:21:41.957 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 Processor -- start 2026-02-22T23:21:41.957 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 -- start start 2026-02-22T23:21:41.957 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 librados: setting wanted keys 2026-02-22T23:21:41.958 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 librados: calling monclient init 2026-02-22T23:21:41.958 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient: init 2026-02-22T23:21:41.958 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:41.958 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:21:41.958 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8004900 0x7fe5bc119690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.958 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): picked mon.noname-b con 0x7fe5a8004900 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:41.958 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe5bc119bd0 0x7fe5bc11a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): picked mon.noname-a con 0x7fe5bc119bd0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5bc1b55b0 0x7fe5bc1b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): picked mon.noname-c con 0x7fe5bc1b55b0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc074330 con 0x7fe5bc119bd0 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc0741b0 con 0x7fe5a8004900 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc121db0 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:41.959 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.943+0000 7fe5c1a58640 10 monclient(hunting): authenticate will time out at 979.626634s 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5bc1b55b0 0x7fe5bc1b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5bc1b55b0 0x7fe5bc1b79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60276/0 (socket says 10.20.193.189:60276) 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 1 -- 10.20.193.189:0/3055741344 learned_addr learned my addr 10.20.193.189:0/3055741344 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 10 monclient(hunting): get_auth_request con 0x7fe5bc1b55b0 auth_method 0 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.944+0000 7fe5bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:41.960 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5baffd640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8004900 0x7fe5bc119690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 10 monclient(hunting): handle_auth_done global_id 4151 payload 824 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 1 -- 10.20.193.189:0/3055741344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8004900 msgr2=0x7fe5bc119690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8004900 0x7fe5bc119690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 1 -- 10.20.193.189:0/3055741344 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe5bc119bd0 msgr2=0x7fe5bc11a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe5bc119bd0 0x7fe5bc11a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 1 monclient: found mon.noname-c 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.961 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.945+0000 7fe5bb7fe640 1 -- 10.20.193.189:0/3055741344 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc1b8210 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.946+0000 7fe5bb7fe640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5bc1b55b0 0x7fe5bc1b79a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc074710 tx=0x7fe5ac00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.946+0000 7fe593fff640 10 client.?.objecter ms_handle_connect 0x7fe5bc1b55b0 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.946+0000 7fe593fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 1 -- 10.20.193.189:0/3055741344 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac017070 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: dump: 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.964 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: _finish_auth 0 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: _check_auth_tickets 2026-02-22T23:21:41.965 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 1 -- 10.20.193.189:0/3055741344 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ac00aa40 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 1 -- 10.20.193.189:0/3055741344 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac007e60 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5b97fa640 4 set_mon_vals no callback set 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe593fff640 10 monclient: dump: 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:41.966 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 5 monclient: authenticate success, global_id 4151 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 monclient: _renew_subs 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc1b84a0 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 client.4151.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 monclient: _renew_subs 2026-02-22T23:21:41.967 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1b89a0 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 1 librados: init done 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.948+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5bc122140 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.954+0000 7fe593fff640 1 -- 10.20.193.189:0/3055741344 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7fe5ac01b3b0 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.955+0000 7fe593fff640 1 -- 10.20.193.189:0/3055741344 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe5ac0113a0 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.955+0000 7fe593fff640 10 client.4151.objecter ms_dispatch2 0x7fe5bc000b90 osd_map(7..7 src has 1..7) 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.955+0000 7fe593fff640 3 client.4151.objecter handle_osd_map got epochs [7,7] > 0 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.955+0000 7fe593fff640 3 client.4151.objecter handle_osd_map decoding full epoch 7 2026-02-22T23:21:41.968 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.955+0000 7fe593fff640 20 client.4151.objecter dump_active .. 0 homeless 2026-02-22T23:21:41.992 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.992+0000 7fe593fff640 1 -- 10.20.193.189:0/3055741344 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7fe5ac021020 con 0x7fe5bc1b55b0 2026-02-22T23:21:41.992 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.992+0000 7fe593fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:41.993 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:41.992+0000 7fe593fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:42.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.024+0000 7fe5c1a58640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "52c5895d-8795-4e32-8b07-c4f246dc0503", "id": 5}] 2026-02-22T23:21:42.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.024+0000 7fe5c1a58640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "52c5895d-8795-4e32-8b07-c4f246dc0503", "id": 5}] 2026-02-22T23:21:42.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.024+0000 7fe5c1a58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:42.024 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.024+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "52c5895d-8795-4e32-8b07-c4f246dc0503", "id": 5} v 0) -- 0x7fe5bc1b8c60 con 0x7fe5bc1b55b0 2026-02-22T23:21:42.131 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe593fff640 1 -- 10.20.193.189:0/3055741344 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "52c5895d-8795-4e32-8b07-c4f246dc0503", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe5ac01b6d0 con 0x7fe5bc1b55b0 2026-02-22T23:21:42.131 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe593fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "52c5895d-8795-4e32-8b07-c4f246dc0503", "id": 5}] 2026-02-22T23:21:42.131 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe593fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:42.131 INFO:teuthology.orchestra.run.trial189.stdout:5 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 10 librados: watch_flush enter 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 10 librados: watch_flush exit 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 20 client.4151.objecter shutdown clearing up homeless session... 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 10 client.4151.objecter successfully canceled tick 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 10 monclient: shutdown 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5bc1b55b0 msgr2=0x7fe5bc1b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5bc1b55b0 0x7fe5bc1b79a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc074710 tx=0x7fe5ac00cce0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 shutdown_connections 2026-02-22T23:21:42.133 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe5bc1b55b0 0x7fe5bc1b79a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.134 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe5bc119bd0 0x7fe5bc11a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.134 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 --2- 10.20.193.189:0/3055741344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe5a8004900 0x7fe5bc119690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.134 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 >> 10.20.193.189:0/3055741344 conn(0x7fe5bc06f640 msgr2=0x7fe5bc1207e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.134 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 shutdown_connections 2026-02-22T23:21:42.134 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 -- 10.20.193.189:0/3055741344 wait complete. 2026-02-22T23:21:42.134 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.131+0000 7fe5c1a58640 1 librados: shutdown 2026-02-22T23:21:42.141 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 6f2925ef-1371-4d00-8ac7-9efd4ef71fe6 6 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.204+0000 7f4ea9405640 10 monclient: get_monmap_and_config 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.204+0000 7f4ea9405640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.204+0000 7f4ea9405640 10 monclient: monmap: 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:42.205386+0000 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:42.205386+0000 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:42.204 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.204+0000 7f4ea9405640 1 Processor -- start 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.204+0000 7f4ea9405640 1 -- start start 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.204+0000 7f4ea9405640 10 monclient: init 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.204+0000 7f4ea9405640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient: _add_conns ranks=[0,5,4,2,3,1] 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4ea4073930 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4ea4117280 0x7f4ea4117710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient(hunting): picked mon.noname-c con 0x7f4ea4117280 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4ea4117c50 0x7f4ea4118100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient(hunting): picked mon.noname-b con 0x7f4ea4117c50 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea4127360 con 0x7f4ea4073930 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea4118780 con 0x7f4ea4117c50 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea4128540 con 0x7f4ea4117280 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea9405640 10 monclient(hunting): authenticate will time out at 979.888057s 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4ea4117280 0x7f4ea4117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4ea4117c50 0x7f4ea4118100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea2ffd640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f4ea4073930 0x7f4ea4073d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.189:53864/0 (socket says 10.20.193.189:53864) 2026-02-22T23:21:42.205 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea2ffd640 1 -- 10.20.193.189:0/1700328220 learned_addr learned my addr 10.20.193.189:0/1700328220 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 10 monclient(hunting): get_auth_request con 0x7f4ea4117280 auth_method 0 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 10 monclient(hunting): get_auth_request con 0x7f4ea4117c50 auth_method 0 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.205+0000 7f4ea27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 10 monclient(hunting): handle_auth_done global_id 4144 payload 274 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 1 -- 10.20.193.189:0/1700328220 >> v2:10.20.193.80:3300/0 conn(0x7f4ea4117280 msgr2=0x7f4ea4117710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:42.206 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 1 --2- 10.20.193.189:0/1700328220 >> v2:10.20.193.80:3300/0 conn(0x7f4ea4117280 0x7f4ea4117710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 1 -- 10.20.193.189:0/1700328220 >> v1:10.20.193.6:6789/0 conn(0x7f4ea4073930 legacy=0x7f4ea4073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 1 monclient: found mon.noname-b 2026-02-22T23:21:42.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.206+0000 7f4ea37fe640 1 -- 10.20.193.189:0/1700328220 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea41286f0 con 0x7f4ea4117c50 2026-02-22T23:21:42.207 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.207+0000 7f4ea37fe640 1 --2- 10.20.193.189:0/1700328220 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4117c50 0x7f4ea4118100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4074420 tx=0x7f4e9402f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b5bb27d94ef221 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 1 -- 10.20.193.189:0/1700328220 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9403c070 con 0x7f4ea4117c50 2026-02-22T23:21:42.209 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: dump: 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea9405640 5 monclient: authenticate success, global_id 4144 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 1 -- 10.20.193.189:0/1700328220 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9402fe40 con 0x7f4ea4117c50 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea9405640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea9405640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 1 -- 10.20.193.189:0/1700328220 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94037660 con 0x7f4ea4117c50 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea9405640 4 set_mon_vals no callback set 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.209+0000 7f4ea1ffb640 10 monclient: dump: 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient: shutdown 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.210 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- 10.20.193.189:0/1700328220 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4117c50 msgr2=0x7f4ea4118100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 --2- 10.20.193.189:0/1700328220 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4117c50 0x7f4ea4118100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4074420 tx=0x7f4e9402f130 comp rx=0 tx=0).stop 2026-02-22T23:21:42.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- 10.20.193.189:0/1700328220 shutdown_connections 2026-02-22T23:21:42.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 --2- 10.20.193.189:0/1700328220 >> v2:10.20.193.80:3300/0 conn(0x7f4ea4117280 0x7f4ea4117710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 --2- 10.20.193.189:0/1700328220 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4117c50 0x7f4ea4118100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- 10.20.193.189:0/1700328220 >> 10.20.193.189:0/1700328220 conn(0x7f4ea406f640 msgr2=0x7f4ea4071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.211 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- 10.20.193.189:0/1700328220 shutdown_connections 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- 10.20.193.189:0/1700328220 wait complete. 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient: monmap: 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 librados: starting msgr at 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 librados: starting objecter 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 Processor -- start 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- start start 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 librados: setting wanted keys 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 librados: calling monclient init 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient: init 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4073930 0x7f4ea41b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): picked mon.noname-b con 0x7f4ea4073930 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ea4117280 0x7f4ea41b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): picked mon.noname-c con 0x7f4ea4117280 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4ea4117c50 0x7f4ea41afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): picked mon.noname-a con 0x7f4ea4117c50 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:42.212 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea41279f0 con 0x7f4ea4117c50 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea4127870 con 0x7f4ea4073930 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea4127b70 con 0x7f4ea4117280 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.210+0000 7f4ea9405640 10 monclient(hunting): authenticate will time out at 979.893915s 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4073930 0x7f4ea41b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ea4117280 0x7f4ea41b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4ea4117c50 0x7f4ea41afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4073930 0x7f4ea41b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.189:52230/0 (socket says 10.20.193.189:52230) 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ea4117280 0x7f4ea41b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60306/0 (socket says 10.20.193.189:60306) 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 1 -- 10.20.193.189:0/572836268 learned_addr learned my addr 10.20.193.189:0/572836268 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 10 monclient(hunting): get_auth_request con 0x7f4ea4073930 auth_method 0 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 10 monclient(hunting): get_auth_request con 0x7f4ea4117280 auth_method 0 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 10 monclient(hunting): get_auth_request con 0x7f4ea4117c50 auth_method 0 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.211+0000 7f4ea37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 10 monclient(hunting): handle_auth_done global_id 4147 payload 824 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 1 -- 10.20.193.189:0/572836268 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ea4117280 msgr2=0x7f4ea41b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 1 --2- 10.20.193.189:0/572836268 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ea4117280 0x7f4ea41b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 1 -- 10.20.193.189:0/572836268 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4ea4117c50 msgr2=0x7f4ea41afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 1 --2- 10.20.193.189:0/572836268 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4ea4117c50 0x7f4ea41afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 1 monclient: found mon.noname-b 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.212+0000 7f4ea2ffd640 1 -- 10.20.193.189:0/572836268 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea41b0330 con 0x7f4ea4073930 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.213+0000 7f4ea2ffd640 1 --2- 10.20.193.189:0/572836268 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4073930 0x7f4ea41b5a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c730 tx=0x7f4e9800cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.213+0000 7f4e837fe640 10 client.?.objecter ms_handle_connect 0x7f4ea4073930 2026-02-22T23:21:42.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.213+0000 7f4e837fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 1 -- 10.20.193.189:0/572836268 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98017070 con 0x7f4ea4073930 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: dump: 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 5 monclient: authenticate success, global_id 4147 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 monclient: _renew_subs 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea41b0620 con 0x7f4ea4073930 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 client.4147.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 monclient: _renew_subs 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea41bc910 con 0x7f4ea4073930 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 1 librados: init done 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 1 -- 10.20.193.189:0/572836268 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e98005b70 con 0x7f4ea4073930 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea17fa640 4 set_mon_vals no callback set 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 1 -- 10.20.193.189:0/572836268 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e98007070 con 0x7f4ea4073930 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4e837fe640 10 monclient: dump: 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.215+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea4117710 con 0x7f4ea4073930 2026-02-22T23:21:42.258 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.257+0000 7f4e837fe640 1 -- 10.20.193.189:0/572836268 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7f4e98007210 con 0x7f4ea4073930 2026-02-22T23:21:42.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 1 -- 10.20.193.189:0/572836268 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f4e9801ade0 con 0x7f4ea4073930 2026-02-22T23:21:42.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 10 client.4147.objecter ms_dispatch2 0x7f4ea4000b90 osd_map(8..8 src has 1..8) 2026-02-22T23:21:42.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 3 client.4147.objecter handle_osd_map got epochs [8,8] > 0 2026-02-22T23:21:42.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 3 client.4147.objecter handle_osd_map decoding full epoch 8 2026-02-22T23:21:42.261 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 20 client.4147.objecter dump_active .. 0 homeless 2026-02-22T23:21:42.262 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 1 -- 10.20.193.189:0/572836268 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f4e98022070 con 0x7f4ea4073930 2026-02-22T23:21:42.262 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.262 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.261+0000 7f4e837fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:42.293 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.293+0000 7f4ea9405640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6f2925ef-1371-4d00-8ac7-9efd4ef71fe6", "id": 6}] 2026-02-22T23:21:42.293 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.293+0000 7f4ea9405640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6f2925ef-1371-4d00-8ac7-9efd4ef71fe6", "id": 6}] 2026-02-22T23:21:42.293 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.293+0000 7f4ea9405640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.293 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.293+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6f2925ef-1371-4d00-8ac7-9efd4ef71fe6", "id": 6} v 0) -- 0x7f4ea4121500 con 0x7f4ea4073930 2026-02-22T23:21:42.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4e837fe640 1 -- 10.20.193.189:0/572836268 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f2925ef-1371-4d00-8ac7-9efd4ef71fe6", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f4e98007710 con 0x7f4ea4073930 2026-02-22T23:21:42.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4e837fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6f2925ef-1371-4d00-8ac7-9efd4ef71fe6", "id": 6}] 2026-02-22T23:21:42.357 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4e837fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:42.358 INFO:teuthology.orchestra.run.trial189.stdout:6 2026-02-22T23:21:42.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 10 librados: watch_flush enter 2026-02-22T23:21:42.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 10 librados: watch_flush exit 2026-02-22T23:21:42.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 20 client.4147.objecter shutdown clearing up homeless session... 2026-02-22T23:21:42.359 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 10 client.4147.objecter successfully canceled tick 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 10 monclient: shutdown 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4073930 msgr2=0x7f4ea41b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 --2- 10.20.193.189:0/572836268 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4073930 0x7f4ea41b5a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c730 tx=0x7f4e9800cc00 comp rx=0 tx=0).stop 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 shutdown_connections 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 --2- 10.20.193.189:0/572836268 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4ea4117c50 0x7f4ea41afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 --2- 10.20.193.189:0/572836268 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ea4117280 0x7f4ea41b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 --2- 10.20.193.189:0/572836268 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ea4073930 0x7f4ea41b5a20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 >> 10.20.193.189:0/572836268 conn(0x7f4ea406f640 msgr2=0x7f4ea4114430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 shutdown_connections 2026-02-22T23:21:42.360 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 -- 10.20.193.189:0/572836268 wait complete. 2026-02-22T23:21:42.361 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.357+0000 7f4ea9405640 1 librados: shutdown 2026-02-22T23:21:42.366 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 963d277c-a350-4d8b-b94f-83365e8c8a9b 7 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient: get_monmap_and_config 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient: monmap: 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:42.431003+0000 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:42.431003+0000 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:42.430 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 1 Processor -- start 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 1 -- start start 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient: init 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient: _add_conns ranks=[5,3,1,4,0,2] 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f16ac073930 0x7f16ac073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient(hunting): picked mon.noname-c con 0x7f16ac073930 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f16ac074250 0x7f16ac11e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient(hunting): picked mon.noname-a con 0x7f16ac074250 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f16ac11eed0 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16ac122c90 con 0x7f16ac11eed0 2026-02-22T23:21:42.431 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f16ac123eb0 con 0x7f16ac074250 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f16ac124030 con 0x7f16ac073930 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.430+0000 7f16b34a9640 10 monclient(hunting): authenticate will time out at 980.113699s 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b1a1f640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f16ac11eed0 0x7f16ac121390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.189:59816/0 (socket says 10.20.193.189:59816) 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b1a1f640 1 -- 10.20.193.189:0/1868401237 learned_addr learned my addr 10.20.193.189:0/1868401237 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 1 --2- 10.20.193.189:0/1868401237 >> v2:10.20.193.80:3300/0 conn(0x7f16ac073930 0x7f16ac073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 10 monclient(hunting): get_auth_request con 0x7f16ac073930 auth_method 0 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.431+0000 7f16b121e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f169bfff640 1 -- 10.20.193.189:0/1868401237 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2562117086 0 0) 0x7f16ac122c90 con 0x7f16ac11eed0 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f169bfff640 10 monclient(hunting): _negotiate 2026-02-22T23:21:42.432 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f169bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f169bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f169bfff640 10 monclient(hunting): my global_id is 4150 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f169bfff640 1 -- 10.20.193.189:0/1868401237 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1694003620 con 0x7f16ac11eed0 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 10 monclient(hunting): handle_auth_done global_id 4160 payload 274 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 1 -- 10.20.193.189:0/1868401237 >> v2:10.20.193.6:3300/0 conn(0x7f16ac074250 msgr2=0x7f16ac11e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 1 --2- 10.20.193.189:0/1868401237 >> v2:10.20.193.6:3300/0 conn(0x7f16ac074250 0x7f16ac11e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 1 -- 10.20.193.189:0/1868401237 >> v1:10.20.193.40:6789/0 conn(0x7f16ac11eed0 legacy=0x7f16ac121390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 1 monclient: found mon.noname-c 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.432+0000 7f16b121e640 1 -- 10.20.193.189:0/1868401237 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac1241e0 con 0x7f16ac073930 2026-02-22T23:21:42.433 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.433+0000 7f16b121e640 1 --2- 10.20.193.189:0/1868401237 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac073930 0x7f16ac073d10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16ac069850 tx=0x7f169c02ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad10a8660cff9e32 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.434 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 1 -- 10.20.193.189:0/1868401237 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c03c070 con 0x7f16ac073930 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: dump: 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 1 -- 10.20.193.189:0/1868401237 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c02fa60 con 0x7f16ac073930 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f16b34a9640 5 monclient: authenticate success, global_id 4160 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 1 -- 10.20.193.189:0/1868401237 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c02fe30 con 0x7f16ac073930 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f16b34a9640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f16b34a9640 4 set_mon_vals no callback set 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.434+0000 7f169bfff640 10 monclient: dump: 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.435 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient: shutdown 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- 10.20.193.189:0/1868401237 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac073930 msgr2=0x7f16ac073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 --2- 10.20.193.189:0/1868401237 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac073930 0x7f16ac073d10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16ac069850 tx=0x7f169c02ed50 comp rx=0 tx=0).stop 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- 10.20.193.189:0/1868401237 shutdown_connections 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 --2- 10.20.193.189:0/1868401237 >> v2:10.20.193.6:3300/0 conn(0x7f16ac074250 0x7f16ac11e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 --2- 10.20.193.189:0/1868401237 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac073930 0x7f16ac073d10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- 10.20.193.189:0/1868401237 >> 10.20.193.189:0/1868401237 conn(0x7f16ac06f640 msgr2=0x7f16ac071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- 10.20.193.189:0/1868401237 shutdown_connections 2026-02-22T23:21:42.436 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- 10.20.193.189:0/1868401237 wait complete. 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient: monmap: 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 librados: starting msgr at 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 librados: starting objecter 2026-02-22T23:21:42.437 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 Processor -- start 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- start start 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 librados: setting wanted keys 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 librados: calling monclient init 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient: init 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16ac073930 0x7f16ac1b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): picked mon.noname-b con 0x7f16ac073930 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac074250 0x7f16ac1b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): picked mon.noname-c con 0x7f16ac074250 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16ac11eed0 0x7f16ac1ab510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): picked mon.noname-a con 0x7f16ac11eed0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac123320 con 0x7f16ac11eed0 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1231a0 con 0x7f16ac073930 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1234a0 con 0x7f16ac074250 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.435+0000 7f16b34a9640 10 monclient(hunting): authenticate will time out at 980.118902s 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac074250 0x7f16ac1b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16ac073930 0x7f16ac1b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac074250 0x7f16ac1b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60326/0 (socket says 10.20.193.189:60326) 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 1 -- 10.20.193.189:0/1751633574 learned_addr learned my addr 10.20.193.189:0/1751633574 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 10 monclient(hunting): get_auth_request con 0x7f16ac074250 auth_method 0 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 10 monclient(hunting): get_auth_request con 0x7f16ac073930 auth_method 0 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.438 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b0a1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.436+0000 7f16b121e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 10 monclient(hunting): handle_auth_done global_id 4163 payload 824 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 1 -- 10.20.193.189:0/1751633574 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16ac073930 msgr2=0x7f16ac1b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 1 --2- 10.20.193.189:0/1751633574 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16ac073930 0x7f16ac1b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 1 -- 10.20.193.189:0/1751633574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16ac11eed0 msgr2=0x7f16ac1ab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 1 --2- 10.20.193.189:0/1751633574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16ac11eed0 0x7f16ac1ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 1 monclient: found mon.noname-c 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.437+0000 7f16b0a1d640 1 -- 10.20.193.189:0/1751633574 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac1abdd0 con 0x7f16ac074250 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.438+0000 7f16b0a1d640 1 --2- 10.20.193.189:0/1751633574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac074250 0x7f16ac1b1ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b570 tx=0x7f16a000ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.438+0000 7f1699ffb640 10 client.?.objecter ms_handle_connect 0x7f16ac074250 2026-02-22T23:21:42.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.438+0000 7f1699ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f1699ffb640 1 -- 10.20.193.189:0/1751633574 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0013020 con 0x7f16ac074250 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f1699ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f1699ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f1699ffb640 10 monclient: dump: 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.440 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f1699ffb640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f1699ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f1699ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 5 monclient: authenticate success, global_id 4163 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 10 monclient: _renew_subs 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac1ac060 con 0x7f16ac074250 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 10 client.4163.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 10 monclient: _renew_subs 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac1b83f0 con 0x7f16ac074250 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.439+0000 7f16b34a9640 1 librados: init done 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f1699ffb640 1 -- 10.20.193.189:0/1751633574 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a0004480 con 0x7f16ac074250 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f1699ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f1699ffb640 1 -- 10.20.193.189:0/1751633574 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a00028e0 con 0x7f16ac074250 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f1699ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f1699ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f1699ffb640 10 monclient: dump: 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.441 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f169b7fe640 4 set_mon_vals no callback set 2026-02-22T23:21:42.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f16b34a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f16b34a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f16b34a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:42.442 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.440+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac073d90 con 0x7f16ac074250 2026-02-22T23:21:42.445 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.446+0000 7f1699ffb640 1 -- 10.20.193.189:0/1751633574 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7f16a0002a80 con 0x7f16ac074250 2026-02-22T23:21:42.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.446+0000 7f1699ffb640 1 -- 10.20.193.189:0/1751633574 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f16a000f460 con 0x7f16ac074250 2026-02-22T23:21:42.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.446+0000 7f1699ffb640 10 client.4163.objecter ms_dispatch2 0x7f16ac000b90 osd_map(9..9 src has 1..9) 2026-02-22T23:21:42.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.447+0000 7f1699ffb640 3 client.4163.objecter handle_osd_map got epochs [9,9] > 0 2026-02-22T23:21:42.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.447+0000 7f1699ffb640 3 client.4163.objecter handle_osd_map decoding full epoch 9 2026-02-22T23:21:42.446 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.447+0000 7f1699ffb640 20 client.4163.objecter dump_active .. 0 homeless 2026-02-22T23:21:42.484 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.483+0000 7f1699ffb640 1 -- 10.20.193.189:0/1751633574 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f16a001d020 con 0x7f16ac074250 2026-02-22T23:21:42.484 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.483+0000 7f1699ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.484 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.483+0000 7f1699ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:42.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.515+0000 7f16b34a9640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "963d277c-a350-4d8b-b94f-83365e8c8a9b", "id": 7}] 2026-02-22T23:21:42.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.515+0000 7f16b34a9640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "963d277c-a350-4d8b-b94f-83365e8c8a9b", "id": 7}] 2026-02-22T23:21:42.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.515+0000 7f16b34a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:42.516 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.515+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "963d277c-a350-4d8b-b94f-83365e8c8a9b", "id": 7} v 0) -- 0x7f16ac1b8620 con 0x7f16ac074250 2026-02-22T23:21:42.570 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f1699ffb640 1 -- 10.20.193.189:0/1751633574 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "963d277c-a350-4d8b-b94f-83365e8c8a9b", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f16a000f6f0 con 0x7f16ac074250 2026-02-22T23:21:42.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f1699ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "963d277c-a350-4d8b-b94f-83365e8c8a9b", "id": 7}] 2026-02-22T23:21:42.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f1699ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:42.571 INFO:teuthology.orchestra.run.trial189.stdout:7 2026-02-22T23:21:42.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 10 librados: watch_flush enter 2026-02-22T23:21:42.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 10 librados: watch_flush exit 2026-02-22T23:21:42.571 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 20 client.4163.objecter shutdown clearing up homeless session... 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 10 client.4163.objecter successfully canceled tick 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 10 monclient: shutdown 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac074250 msgr2=0x7f16ac1b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 --2- 10.20.193.189:0/1751633574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac074250 0x7f16ac1b1ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b570 tx=0x7f16a000ba40 comp rx=0 tx=0).stop 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 shutdown_connections 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 --2- 10.20.193.189:0/1751633574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16ac11eed0 0x7f16ac1ab510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 --2- 10.20.193.189:0/1751633574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16ac074250 0x7f16ac1b1ad0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 --2- 10.20.193.189:0/1751633574 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16ac073930 0x7f16ac1b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 >> 10.20.193.189:0/1751633574 conn(0x7f16ac06f640 msgr2=0x7f16ac11f380 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 shutdown_connections 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 -- 10.20.193.189:0/1751633574 wait complete. 2026-02-22T23:21:42.572 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.571+0000 7f16b34a9640 1 librados: shutdown 2026-02-22T23:21:42.578 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c 8 2026-02-22T23:21:42.644 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient: get_monmap_and_config 2026-02-22T23:21:42.644 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.644 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient: monmap: 2026-02-22T23:21:42.644 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.644 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.644 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:42.641960+0000 2026-02-22T23:21:42.644 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:42.641960+0000 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 1 Processor -- start 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 1 -- start start 2026-02-22T23:21:42.645 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient: init 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient: _add_conns ranks=[2,5,3,1,4,0] 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f737c073930 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f737c11d960 0x7f737c11fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient(hunting): picked mon.noname-c con 0x7f737c11d960 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f737c120290 0x7f737c122680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient(hunting): picked mon.noname-a con 0x7f737c120290 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f737c122c80 con 0x7f737c073930 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f737c123ea0 con 0x7f737c120290 2026-02-22T23:21:42.646 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f737c124020 con 0x7f737c11d960 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.641+0000 7f7382140640 10 monclient(hunting): authenticate will time out at 980.324643s 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f737c120290 0x7f737c122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737b7fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f737c073930 0x7f737c073d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.189:56918/0 (socket says 10.20.193.189:56918) 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737b7fe640 1 -- 10.20.193.189:0/128794020 learned_addr learned my addr 10.20.193.189:0/128794020 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 10 monclient(hunting): get_auth_request con 0x7f737c120290 auth_method 0 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737affd640 1 --2- 10.20.193.189:0/128794020 >> v2:10.20.193.80:3300/0 conn(0x7f737c11d960 0x7f737c11fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.647 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737affd640 10 monclient(hunting): get_auth_request con 0x7f737c11d960 auth_method 0 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.642+0000 7f737affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 10 monclient(hunting): handle_auth_done global_id 4149 payload 274 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 1 -- 10.20.193.189:0/128794020 >> v2:10.20.193.80:3300/0 conn(0x7f737c11d960 msgr2=0x7f737c11fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:42.648 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 1 --2- 10.20.193.189:0/128794020 >> v2:10.20.193.80:3300/0 conn(0x7f737c11d960 0x7f737c11fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 1 -- 10.20.193.189:0/128794020 >> v1:10.20.193.80:6789/0 conn(0x7f737c073930 legacy=0x7f737c073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 1 monclient: found mon.noname-a 2026-02-22T23:21:42.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:42.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.643+0000 7f737bfff640 1 -- 10.20.193.189:0/128794020 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c1241d0 con 0x7f737c120290 2026-02-22T23:21:42.649 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.644+0000 7f737bfff640 1 --2- 10.20.193.189:0/128794020 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c120290 0x7f737c122680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f737c114250 tx=0x7f737002f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebafd29119bd8dc server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.656 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 1 -- 10.20.193.189:0/128794020 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737003c070 con 0x7f737c120290 2026-02-22T23:21:42.656 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.656 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:42.656 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: dump: 2026-02-22T23:21:42.656 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.656 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.656 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.657 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f7382140640 5 monclient: authenticate success, global_id 4149 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 1 -- 10.20.193.189:0/128794020 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737002fe40 con 0x7f737c120290 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f7382140640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f7382140640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 1 -- 10.20.193.189:0/128794020 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370037660 con 0x7f737c120290 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f7382140640 4 set_mon_vals no callback set 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.645+0000 7f737a7fc640 10 monclient: dump: 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.658 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient: shutdown 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- 10.20.193.189:0/128794020 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c120290 msgr2=0x7f737c122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 --2- 10.20.193.189:0/128794020 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c120290 0x7f737c122680 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f737c114250 tx=0x7f737002f130 comp rx=0 tx=0).stop 2026-02-22T23:21:42.659 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- 10.20.193.189:0/128794020 shutdown_connections 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 --2- 10.20.193.189:0/128794020 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c120290 0x7f737c122680 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 --2- 10.20.193.189:0/128794020 >> v2:10.20.193.80:3300/0 conn(0x7f737c11d960 0x7f737c11fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- 10.20.193.189:0/128794020 >> 10.20.193.189:0/128794020 conn(0x7f737c06f640 msgr2=0x7f737c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- 10.20.193.189:0/128794020 shutdown_connections 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- 10.20.193.189:0/128794020 wait complete. 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient: monmap: 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.660 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 librados: starting msgr at 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 librados: starting objecter 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 Processor -- start 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- start start 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 librados: setting wanted keys 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 librados: calling monclient init 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient: init 2026-02-22T23:21:42.661 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c073930 0x7f737c1b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): picked mon.noname-a con 0x7f737c073930 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f737c11d960 0x7f737c1b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): picked mon.noname-b con 0x7f737c11d960 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f737c120290 0x7f737c1ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): picked mon.noname-c con 0x7f737c120290 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f737c123310 con 0x7f737c073930 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f737c123190 con 0x7f737c11d960 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.662 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f737c123490 con 0x7f737c120290 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.646+0000 7f7382140640 10 monclient(hunting): authenticate will time out at 980.329950s 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c073930 0x7f737c1b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f737c120290 0x7f737c1ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f737c11d960 0x7f737c1b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c073930 0x7f737c1b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.189:36952/0 (socket says 10.20.193.189:36952) 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f737c120290 0x7f737c1ab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60350/0 (socket says 10.20.193.189:60350) 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 1 -- 10.20.193.189:0/654195647 learned_addr learned my addr 10.20.193.189:0/654195647 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 10 monclient(hunting): get_auth_request con 0x7f737c073930 auth_method 0 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.663 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 10 monclient(hunting): get_auth_request con 0x7f737c120290 auth_method 0 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 10 monclient(hunting): get_auth_request con 0x7f737c11d960 auth_method 0 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.664 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.647+0000 7f737affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 10 monclient(hunting): handle_auth_done global_id 4152 payload 824 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 1 -- 10.20.193.189:0/654195647 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f737c120290 msgr2=0x7f737c1ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 1 --2- 10.20.193.189:0/654195647 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f737c120290 0x7f737c1ab4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 1 -- 10.20.193.189:0/654195647 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f737c11d960 msgr2=0x7f737c1b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 1 --2- 10.20.193.189:0/654195647 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f737c11d960 0x7f737c1b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:21:42.665 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.648+0000 7f737b7fe640 1 -- 10.20.193.189:0/654195647 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c1abd80 con 0x7f737c073930 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.649+0000 7f737b7fe640 1 --2- 10.20.193.189:0/654195647 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c073930 0x7f737c1b1410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f736800c700 tx=0x7f736800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.649+0000 7f7357fff640 10 client.?.objecter ms_handle_connect 0x7f737c073930 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.649+0000 7f7357fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 1 -- 10.20.193.189:0/654195647 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368017070 con 0x7f737c073930 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: dump: 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.666 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 1 -- 10.20.193.189:0/654195647 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7368005bb0 con 0x7f737c073930 2026-02-22T23:21:42.667 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 5 monclient: authenticate success, global_id 4152 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 10 monclient: _renew_subs 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c1ac010 con 0x7f737c073930 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 1 -- 10.20.193.189:0/654195647 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368007070 con 0x7f737c073930 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7379ffb640 4 set_mon_vals no callback set 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7357fff640 10 monclient: dump: 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.668 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 10 client.4152.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 10 monclient: _renew_subs 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c1b83a0 con 0x7f737c073930 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.651+0000 7f7382140640 1 librados: init done 2026-02-22T23:21:42.669 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.652+0000 7f7382140640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.670 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.652+0000 7f7382140640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.670 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.652+0000 7f7382140640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:42.670 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.652+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c1238a0 con 0x7f737c073930 2026-02-22T23:21:42.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.657+0000 7f7357fff640 1 -- 10.20.193.189:0/654195647 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7f7368007210 con 0x7f737c073930 2026-02-22T23:21:42.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.658+0000 7f7357fff640 1 -- 10.20.193.189:0/654195647 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7368010d80 con 0x7f737c073930 2026-02-22T23:21:42.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.658+0000 7f7357fff640 10 client.4152.objecter ms_dispatch2 0x7f737c000b90 osd_map(10..10 src has 1..10) 2026-02-22T23:21:42.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.658+0000 7f7357fff640 3 client.4152.objecter handle_osd_map got epochs [10,10] > 0 2026-02-22T23:21:42.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.658+0000 7f7357fff640 3 client.4152.objecter handle_osd_map decoding full epoch 10 2026-02-22T23:21:42.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.658+0000 7f7357fff640 20 client.4152.objecter dump_active .. 0 homeless 2026-02-22T23:21:42.698 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.698+0000 7f7357fff640 1 -- 10.20.193.189:0/654195647 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f7368022020 con 0x7f737c073930 2026-02-22T23:21:42.698 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.698+0000 7f7357fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.698 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.698+0000 7f7357fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:42.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.731+0000 7f7382140640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c", "id": 8}] 2026-02-22T23:21:42.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.731+0000 7f7382140640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c", "id": 8}] 2026-02-22T23:21:42.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.731+0000 7f7382140640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:42.731 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.731+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c", "id": 8} v 0) -- 0x7f737c0722c0 con 0x7f737c073930 2026-02-22T23:21:42.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.748+0000 7f7357fff640 1 -- 10.20.193.189:0/654195647 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c", "id": 8}]=0 v11) ==== 112+0+2 (secure 0 0 0) 0x7f737c0722c0 con 0x7f737c073930 2026-02-22T23:21:42.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.748+0000 7f7357fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c", "id": 8}] 2026-02-22T23:21:42.748 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.748+0000 7f7357fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:42.749 INFO:teuthology.orchestra.run.trial189.stdout:8 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 10 librados: watch_flush enter 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 10 librados: watch_flush exit 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 20 client.4152.objecter shutdown clearing up homeless session... 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 10 client.4152.objecter successfully canceled tick 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 10 monclient: shutdown 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c073930 msgr2=0x7f737c1b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 --2- 10.20.193.189:0/654195647 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c073930 0x7f737c1b1410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f736800c700 tx=0x7f736800cbd0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 shutdown_connections 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 --2- 10.20.193.189:0/654195647 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f737c120290 0x7f737c1ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.750 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 --2- 10.20.193.189:0/654195647 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f737c11d960 0x7f737c1b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 --2- 10.20.193.189:0/654195647 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f737c073930 0x7f737c1b1410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 >> 10.20.193.189:0/654195647 conn(0x7f737c06f640 msgr2=0x7f737c06fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 shutdown_connections 2026-02-22T23:21:42.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 -- 10.20.193.189:0/654195647 wait complete. 2026-02-22T23:21:42.751 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.749+0000 7f7382140640 1 librados: shutdown 2026-02-22T23:21:42.755 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 528c9e90-8c81-4193-a32a-6809648cf450 9 2026-02-22T23:21:42.818 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient: get_monmap_and_config 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient: monmap: 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:42.819934+0000 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:42.819934+0000 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:42.819 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 1 Processor -- start 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 1 -- start start 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient: init 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient: _add_conns ranks=[5,2,4,3,0,1] 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f7d18073930 0x7f7d18073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient(hunting): picked mon.noname-c con 0x7f7d18073930 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7d18074280 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7d18118410 0x7f7d18116930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient(hunting): picked mon.noname-b con 0x7f7d18118410 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d18127360 con 0x7f7d18074280 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7d18117050 con 0x7f7d18118410 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7d18128540 con 0x7f7d18073930 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d205c3640 10 monclient(hunting): authenticate will time out at 980.502604s 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.819+0000 7f7d1eb39640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7d18118410 0x7f7d18116930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f7d18073930 0x7f7d18073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7d18118410 0x7f7d18116930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.189:52248/0 (socket says 10.20.193.189:52248) 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 1 -- 10.20.193.189:0/3839896930 learned_addr learned my addr 10.20.193.189:0/3839896930 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 10 monclient(hunting): get_auth_request con 0x7f7d18118410 auth_method 0 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 10 monclient(hunting): get_auth_request con 0x7f7d18073930 auth_method 0 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.820 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.820+0000 7f7d1e338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_done global_id 4159 payload 274 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 1 -- 10.20.193.189:0/3839896930 >> v2:10.20.193.80:3300/0 conn(0x7f7d18073930 msgr2=0x7f7d18073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 1 --2- 10.20.193.189:0/3839896930 >> v2:10.20.193.80:3300/0 conn(0x7f7d18073930 0x7f7d18073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 1 -- 10.20.193.189:0/3839896930 >> v1:10.20.193.80:6789/0 conn(0x7f7d18074280 legacy=0x7f7d18116200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 1 monclient: found mon.noname-b 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 1 -- 10.20.193.189:0/3839896930 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d181286f0 con 0x7f7d18118410 2026-02-22T23:21:42.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.821+0000 7f7d1eb39640 1 --2- 10.20.193.189:0/3839896930 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d18116930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d18069850 tx=0x7f7d1402f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee87a3185be6e01 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 1 -- 10.20.193.189:0/3839896930 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1402fcf0 con 0x7f7d18118410 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: dump: 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 1 -- 10.20.193.189:0/3839896930 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d140373f0 con 0x7f7d18118410 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 5 monclient: authenticate success, global_id 4159 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 1 -- 10.20.193.189:0/3839896930 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140377c0 con 0x7f7d18118410 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d1d336640 10 monclient: dump: 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 4 set_mon_vals no callback set 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 10 monclient: shutdown 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:42.824 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 -- 10.20.193.189:0/3839896930 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 msgr2=0x7f7d18116930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 --2- 10.20.193.189:0/3839896930 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d18116930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7d18069850 tx=0x7f7d1402f230 comp rx=0 tx=0).stop 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 -- 10.20.193.189:0/3839896930 shutdown_connections 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 --2- 10.20.193.189:0/3839896930 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d18116930 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 --2- 10.20.193.189:0/3839896930 >> v2:10.20.193.80:3300/0 conn(0x7f7d18073930 0x7f7d18073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 -- 10.20.193.189:0/3839896930 >> 10.20.193.189:0/3839896930 conn(0x7f7d1806f640 msgr2=0x7f7d18071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 -- 10.20.193.189:0/3839896930 shutdown_connections 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.823+0000 7f7d205c3640 1 -- 10.20.193.189:0/3839896930 wait complete. 2026-02-22T23:21:42.825 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient: build_initial_monmap 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient: monmap: 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 librados: starting msgr at 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 librados: starting objecter 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 Processor -- start 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 -- start start 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 librados: setting wanted keys 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 librados: calling monclient init 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient: init 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d18073930 0x7f7d181afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): picked mon.noname-c con 0x7f7d18073930 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d18074280 0x7f7d181b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): picked mon.noname-a con 0x7f7d18074280 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d181b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): picked mon.noname-b con 0x7f7d18118410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18127990 con 0x7f7d18074280 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18127810 con 0x7f7d18118410 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d18127b10 con 0x7f7d18073930 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.824+0000 7f7d205c3640 10 monclient(hunting): authenticate will time out at 980.507761s 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d181b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d18074280 0x7f7d181b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d181b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.189:52252/0 (socket says 10.20.193.189:52252) 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 1 -- 10.20.193.189:0/1549523246 learned_addr learned my addr 10.20.193.189:0/1549523246 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 10 monclient(hunting): get_auth_request con 0x7f7d18118410 auth_method 0 2026-02-22T23:21:42.826 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 10 monclient(hunting): get_auth_request con 0x7f7d18074280 auth_method 0 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1e338640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d18073930 0x7f7d181afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1db37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1e338640 10 monclient(hunting): get_auth_request con 0x7f7d18073930 auth_method 0 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1e338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1e338640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.825+0000 7f7d1e338640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1e338640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1e338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1e338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 10 monclient(hunting): handle_auth_done global_id 4162 payload 824 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 1 -- 10.20.193.189:0/1549523246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d18073930 msgr2=0x7f7d181afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d18073930 0x7f7d181afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 1 -- 10.20.193.189:0/1549523246 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d18074280 msgr2=0x7f7d181b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d18074280 0x7f7d181b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:42.827 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 10 monclient: _finish_hunting 0 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 1 monclient: found mon.noname-b 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.826+0000 7f7d1eb39640 1 -- 10.20.193.189:0/1549523246 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d181b4d20 con 0x7f7d18118410 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.827+0000 7f7d1eb39640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d181b4460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d14039040 tx=0x7f7d14004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.827+0000 7f7cfeffd640 10 client.?.objecter ms_handle_connect 0x7f7d18118410 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.827+0000 7f7cfeffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14046020 con 0x7f7d18118410 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 10 monclient: dump: 2026-02-22T23:21:42.828 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 10 monclient: _finish_auth 0 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 5 monclient: authenticate success, global_id 4162 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 10 monclient: _renew_subs 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d181b4fb0 con 0x7f7d18118410 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 10 client.4162.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 10 monclient: _renew_subs 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d181bc790 con 0x7f7d18118410 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d205c3640 1 librados: init done 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d140373f0 con 0x7f7d18118410 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7cfeffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.828+0000 7f7d1cb35640 4 set_mon_vals no callback set 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14037e60 con 0x7f7d18118410 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7cfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7cfeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7cfeffd640 10 monclient: dump: 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:42.829 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:42.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7d205c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7d205c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7d205c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.830 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.829+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d18073d10 con 0x7f7d18118410 2026-02-22T23:21:42.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.835+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 2) ==== 49971+0+0 (secure 0 0 0) 0x7f7d14041650 con 0x7f7d18118410 2026-02-22T23:21:42.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.835+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1760+0+0 (secure 0 0 0) 0x7f7d140533a0 con 0x7f7d18118410 2026-02-22T23:21:42.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.835+0000 7f7cfeffd640 10 client.4162.objecter ms_dispatch2 0x7f7d18000b90 osd_map(11..11 src has 1..11) 2026-02-22T23:21:42.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.835+0000 7f7cfeffd640 3 client.4162.objecter handle_osd_map got epochs [11,11] > 0 2026-02-22T23:21:42.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.835+0000 7f7cfeffd640 3 client.4162.objecter handle_osd_map decoding full epoch 11 2026-02-22T23:21:42.835 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.835+0000 7f7cfeffd640 20 client.4162.objecter dump_active .. 0 homeless 2026-02-22T23:21:42.874 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.875+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75931 (secure 0 0 0) 0x7f7d14050030 con 0x7f7d18118410 2026-02-22T23:21:42.874 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.875+0000 7f7cfeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:42.875 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.875+0000 7f7cfeffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:42.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.906+0000 7f7d205c3640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "528c9e90-8c81-4193-a32a-6809648cf450", "id": 9}] 2026-02-22T23:21:42.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.906+0000 7f7d205c3640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "528c9e90-8c81-4193-a32a-6809648cf450", "id": 9}] 2026-02-22T23:21:42.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.906+0000 7f7d205c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:42.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:42.906+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "528c9e90-8c81-4193-a32a-6809648cf450", "id": 9} v 0) -- 0x7f7d181bc9b0 con 0x7f7d18118410 2026-02-22T23:21:43.058 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.058+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f7d1404bcb0 con 0x7f7d18118410 2026-02-22T23:21:43.059 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.059+0000 7f7cfeffd640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7ce80794f0 0x7f7ce807b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.060 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.059+0000 7f7d1e338640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7ce80794f0 0x7f7ce807b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.060 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.059+0000 7f7d1e338640 10 monclient: get_auth_request con 0x7f7ce80794f0 auth_method 0 2026-02-22T23:21:43.060 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.059+0000 7f7cfeffd640 10 client.4162.objecter ms_handle_connect 0x7f7ce80794f0 2026-02-22T23:21:43.060 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.059+0000 7f7d1e338640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7ce80794f0 0x7f7ce807b8e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009c40 tx=0x7f7d08009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.093 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:43.093+0000 7f7292494640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-22T23:21:43.095 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:21:43.095+0000 7f7292494640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-22T23:21:43.127 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7cfeffd640 1 -- 10.20.193.189:0/1549523246 <== mon.1 v2:10.20.193.40:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "528c9e90-8c81-4193-a32a-6809648cf450", "id": 9}]=0 v12) ==== 112+0+2 (secure 0 0 0) 0x7f7d140403b0 con 0x7f7d18118410 2026-02-22T23:21:43.127 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7cfeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "528c9e90-8c81-4193-a32a-6809648cf450", "id": 9}] 2026-02-22T23:21:43.127 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7cfeffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:43.128 INFO:teuthology.orchestra.run.trial189.stdout:9 2026-02-22T23:21:43.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 10 librados: watch_flush enter 2026-02-22T23:21:43.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 10 librados: watch_flush exit 2026-02-22T23:21:43.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 20 client.4162.objecter shutdown clearing up homeless session... 2026-02-22T23:21:43.129 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 10 client.4162.objecter successfully canceled tick 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7ce80794f0 msgr2=0x7f7ce807b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7ce80794f0 0x7f7ce807b8e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009c40 tx=0x7f7d08009290 comp rx=0 tx=0).stop 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 10 monclient: shutdown 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 msgr2=0x7f7d181b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d181b4460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7d14039040 tx=0x7f7d14004b40 comp rx=0 tx=0).stop 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 shutdown_connections 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7ce80794f0 0x7f7ce807b8e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d18118410 0x7f7d181b4460 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d18074280 0x7f7d181b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 --2- 10.20.193.189:0/1549523246 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d18073930 0x7f7d181afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.130 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 >> 10.20.193.189:0/1549523246 conn(0x7f7d1806f640 msgr2=0x7f7d18070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:43.131 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 shutdown_connections 2026-02-22T23:21:43.131 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 -- 10.20.193.189:0/1549523246 wait complete. 2026-02-22T23:21:43.131 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.127+0000 7f7d205c3640 1 librados: shutdown 2026-02-22T23:21:43.135 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new acd57453-2064-44a4-bb36-a8c2d2c13c5b 10 2026-02-22T23:21:43.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient: get_monmap_and_config 2026-02-22T23:21:43.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient: build_initial_monmap 2026-02-22T23:21:43.201 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient: monmap: 2026-02-22T23:21:43.201 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:43.201 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.201 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:43.199908+0000 2026-02-22T23:21:43.201 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:43.199908+0000 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 1 Processor -- start 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 1 -- start start 2026-02-22T23:21:43.202 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient: init 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient: _add_conns ranks=[0,5,2,3,4,1] 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7faa50073930 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7faa50117280 0x7faa50117710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient(hunting): picked mon.noname-c con 0x7faa50117280 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7faa50117c50 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa50128400 con 0x7faa50073930 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa501295e0 con 0x7faa50117c50 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7faa5012a7c0 con 0x7faa50117280 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:43.203 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.199+0000 7faa58387640 10 monclient(hunting): authenticate will time out at 980.882596s 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7faa50117280 0x7faa50117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa560fc640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7faa50073930 0x7faa50073d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.189:53866/0 (socket says 10.20.193.189:53866) 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa560fc640 1 -- 10.20.193.189:0/3432493985 learned_addr learned my addr 10.20.193.189:0/3432493985 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 10 monclient(hunting): get_auth_request con 0x7faa50117280 auth_method 0 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.204 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.200+0000 7faa558fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa550fa640 1 -- 10.20.193.189:0/3432493985 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061477863 0 0) 0x7faa50128400 con 0x7faa50073930 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa550fa640 10 monclient(hunting): _negotiate 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa550fa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa550fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa550fa640 10 monclient(hunting): my global_id is 4158 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa550fa640 1 -- 10.20.193.189:0/3432493985 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa38003620 con 0x7faa50073930 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 10 monclient(hunting): handle_auth_done global_id 4184 payload 274 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 1 -- 10.20.193.189:0/3432493985 >> v1:10.20.193.80:6789/0 conn(0x7faa50117c50 legacy=0x7faa50118100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.213 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 1 -- 10.20.193.189:0/3432493985 >> v1:10.20.193.6:6789/0 conn(0x7faa50073930 legacy=0x7faa50073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 10 monclient: _finish_hunting 0 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 1 monclient: found mon.noname-c 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.201+0000 7faa558fb640 1 -- 10.20.193.189:0/3432493985 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5012a940 con 0x7faa50117280 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.204+0000 7faa558fb640 1 --2- 10.20.193.189:0/3432493985 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117280 0x7faa50117710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa4402edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7208c248f1f54151 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 1 -- 10.20.193.189:0/3432493985 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4403c070 con 0x7faa50117280 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: dump: 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.214 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: _finish_auth 0 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: _check_auth_tickets 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa58387640 5 monclient: authenticate success, global_id 4184 2026-02-22T23:21:43.215 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa58387640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 1 -- 10.20.193.189:0/3432493985 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa4402fae0 con 0x7faa50117280 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 1 -- 10.20.193.189:0/3432493985 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44037440 con 0x7faa50117280 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa550fa640 10 monclient: dump: 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.216 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa58387640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.205+0000 7faa58387640 4 set_mon_vals no callback set 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient: shutdown 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- 10.20.193.189:0/3432493985 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117280 msgr2=0x7faa50117710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 --2- 10.20.193.189:0/3432493985 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117280 0x7faa50117710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faa440098a0 tx=0x7faa4402edd0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- 10.20.193.189:0/3432493985 shutdown_connections 2026-02-22T23:21:43.217 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 --2- 10.20.193.189:0/3432493985 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117280 0x7faa50117710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- 10.20.193.189:0/3432493985 >> 10.20.193.189:0/3432493985 conn(0x7faa5006f640 msgr2=0x7faa50071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- 10.20.193.189:0/3432493985 shutdown_connections 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- 10.20.193.189:0/3432493985 wait complete. 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient: build_initial_monmap 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient: monmap: 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.218 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 librados: starting msgr at 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 librados: starting objecter 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 Processor -- start 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- start start 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 librados: setting wanted keys 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 librados: calling monclient init 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient: init 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:21:43.219 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7faa50073930 0x7faa50126860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): picked mon.noname-b con 0x7faa50073930 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7faa50117280 0x7faa50126ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): picked mon.noname-a con 0x7faa50117280 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117c50 0x7faa501bb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): picked mon.noname-c con 0x7faa50117c50 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7faa50129c30 con 0x7faa50117280 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7faa50129ab0 con 0x7faa50073930 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7faa50129db0 con 0x7faa50117c50 2026-02-22T23:21:43.220 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.206+0000 7faa58387640 10 monclient(hunting): authenticate will time out at 980.889857s 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7faa50117280 0x7faa50126ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117c50 0x7faa501bb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7faa50073930 0x7faa50126860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117c50 0x7faa501bb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60390/0 (socket says 10.20.193.189:60390) 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 1 -- 10.20.193.189:0/4106816228 learned_addr learned my addr 10.20.193.189:0/4106816228 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 10 monclient(hunting): get_auth_request con 0x7faa50117c50 auth_method 0 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 10 monclient(hunting): get_auth_request con 0x7faa50117280 auth_method 0 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.221 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 10 monclient(hunting): get_auth_request con 0x7faa50073930 auth_method 0 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa568fd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa558fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.222 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.207+0000 7faa560fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 10 monclient(hunting): handle_auth_done global_id 4190 payload 824 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 1 -- 10.20.193.189:0/4106816228 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7faa50073930 msgr2=0x7faa50126860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7faa50073930 0x7faa50126860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 1 -- 10.20.193.189:0/4106816228 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7faa50117280 msgr2=0x7faa50126ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7faa50117280 0x7faa50126ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 10 monclient: _finish_hunting 0 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 1 monclient: found mon.noname-c 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.208+0000 7faa568fd640 1 -- 10.20.193.189:0/4106816228 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa501bbff0 con 0x7faa50117c50 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.209+0000 7faa568fd640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117c50 0x7faa501bb7d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00cc50 tx=0x7faa4c00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.223 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.209+0000 7faa36ffd640 10 client.?.objecter ms_handle_connect 0x7faa50117c50 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.209+0000 7faa36ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa36ffd640 1 -- 10.20.193.189:0/4106816228 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c003a70 con 0x7faa50117c50 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa36ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa36ffd640 10 monclient: dump: 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.224 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa36ffd640 10 monclient: _finish_auth 0 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa36ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa36ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 5 monclient: authenticate success, global_id 4190 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 10 monclient: _renew_subs 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa501bc280 con 0x7faa50117c50 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 10 client.4190.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 10 monclient: _renew_subs 2026-02-22T23:21:43.225 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501bc870 con 0x7faa50117c50 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.210+0000 7faa58387640 1 librados: init done 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa58387640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa58387640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa58387640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa50117790 con 0x7faa50117c50 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa36ffd640 1 -- 10.20.193.189:0/4106816228 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa4c003c10 con 0x7faa50117c50 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa36ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa36ffd640 1 -- 10.20.193.189:0/4106816228 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0050d0 con 0x7faa50117c50 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa36ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa36ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa36ffd640 10 monclient: dump: 2026-02-22T23:21:43.226 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.227 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.211+0000 7faa548f9640 4 set_mon_vals no callback set 2026-02-22T23:21:43.316 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.316+0000 7faa36ffd640 1 -- 10.20.193.189:0/4106816228 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7faa4c005270 con 0x7faa50117c50 2026-02-22T23:21:43.317 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.317+0000 7faa36ffd640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7faa20078c50 0x7faa2007b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.318 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.317+0000 7faa560fc640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7faa20078c50 0x7faa2007b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.318 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.317+0000 7faa560fc640 10 monclient: get_auth_request con 0x7faa20078c50 auth_method 0 2026-02-22T23:21:43.318 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.317+0000 7faa560fc640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7faa20078c50 0x7faa2007b110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa40005980 tx=0x7faa40007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.318 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.317+0000 7faa36ffd640 10 client.4190.objecter ms_handle_connect 0x7faa20078c50 2026-02-22T23:21:43.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.323+0000 7faa36ffd640 1 -- 10.20.193.189:0/4106816228 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 1875+0+0 (secure 0 0 0) 0x7faa4c05e710 con 0x7faa50117c50 2026-02-22T23:21:43.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.323+0000 7faa36ffd640 10 client.4190.objecter ms_dispatch2 0x7faa50000b90 osd_map(12..12 src has 1..12) 2026-02-22T23:21:43.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.323+0000 7faa36ffd640 3 client.4190.objecter handle_osd_map got epochs [12,12] > 0 2026-02-22T23:21:43.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.323+0000 7faa36ffd640 3 client.4190.objecter handle_osd_map decoding full epoch 12 2026-02-22T23:21:43.324 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.324+0000 7faa36ffd640 20 client.4190.objecter dump_active .. 0 homeless 2026-02-22T23:21:43.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.324+0000 7faa36ffd640 1 -- 10.20.193.189:0/4106816228 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7faa4c0626d0 con 0x7faa50117c50 2026-02-22T23:21:43.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.324+0000 7faa36ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.325 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.324+0000 7faa36ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:43.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.410+0000 7faa58387640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "acd57453-2064-44a4-bb36-a8c2d2c13c5b", "id": 10}] 2026-02-22T23:21:43.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.410+0000 7faa58387640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "acd57453-2064-44a4-bb36-a8c2d2c13c5b", "id": 10}] 2026-02-22T23:21:43.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.410+0000 7faa58387640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:43.410 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.410+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "acd57453-2064-44a4-bb36-a8c2d2c13c5b", "id": 10} v 0) -- 0x7faa50115330 con 0x7faa50117c50 2026-02-22T23:21:43.504 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.504+0000 7faa36ffd640 1 -- 10.20.193.189:0/4106816228 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "acd57453-2064-44a4-bb36-a8c2d2c13c5b", "id": 10}]=0 v13) ==== 113+0+3 (secure 0 0 0) 0x7faa4c0669d0 con 0x7faa50117c50 2026-02-22T23:21:43.505 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.504+0000 7faa36ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "acd57453-2064-44a4-bb36-a8c2d2c13c5b", "id": 10}] 2026-02-22T23:21:43.505 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.504+0000 7faa36ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:43.505 INFO:teuthology.orchestra.run.trial189.stdout:10 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 10 librados: watch_flush enter 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 10 librados: watch_flush exit 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 20 client.4190.objecter shutdown clearing up homeless session... 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 10 client.4190.objecter successfully canceled tick 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7faa20078c50 msgr2=0x7faa2007b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7faa20078c50 0x7faa2007b110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faa40005980 tx=0x7faa40007dd0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 10 monclient: shutdown 2026-02-22T23:21:43.507 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117c50 msgr2=0x7faa501bb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117c50 0x7faa501bb7d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00cc50 tx=0x7faa4c00a430 comp rx=0 tx=0).stop 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 shutdown_connections 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7faa20078c50 0x7faa2007b110 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7faa50117c50 0x7faa501bb7d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7faa50117280 0x7faa50126ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 --2- 10.20.193.189:0/4106816228 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7faa50073930 0x7faa50126860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 >> 10.20.193.189:0/4106816228 conn(0x7faa5006f640 msgr2=0x7faa50070590 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 shutdown_connections 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 -- 10.20.193.189:0/4106816228 wait complete. 2026-02-22T23:21:43.508 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.505+0000 7faa58387640 1 librados: shutdown 2026-02-22T23:21:43.515 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new ea741f02-ae0b-419b-8892-34f4b28f6ab6 11 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.580+0000 7f4b7f5a0640 10 monclient: get_monmap_and_config 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.580+0000 7f4b7f5a0640 10 monclient: build_initial_monmap 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.580+0000 7f4b7f5a0640 10 monclient: monmap: 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:43.581666+0000 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:43.581666+0000 2026-02-22T23:21:43.580 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 1 Processor -- start 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 1 -- start start 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient: init 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient: _add_conns ranks=[5,1,3,4,0,2] 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4b78073930 0x7f4b78073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.581 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient(hunting): picked mon.noname-c con 0x7f4b78073930 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4b78074250 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f4b781200b0 0x7f4b78122570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient(hunting): picked mon.noname-a con 0x7f4b781200b0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b78122c90 con 0x7f4b78074250 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4b78123eb0 con 0x7f4b781200b0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4b78124030 con 0x7f4b78073930 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7f5a0640 10 monclient(hunting): authenticate will time out at 981.264373s 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7d315640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4b78073930 0x7f4b78073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7cb14640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f4b78074250 0x7f4b7811e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.189:59820/0 (socket says 10.20.193.189:59820) 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7cb14640 1 -- 10.20.193.189:0/2514659034 learned_addr learned my addr 10.20.193.189:0/2514659034 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7db16640 1 --2- 10.20.193.189:0/2514659034 >> v2:10.20.193.6:3300/0 conn(0x7f4b781200b0 0x7f4b78122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7d315640 10 monclient(hunting): get_auth_request con 0x7f4b78073930 auth_method 0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7d315640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7d315640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.581+0000 7f4b7d315640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7db16640 10 monclient(hunting): get_auth_request con 0x7f4b781200b0 auth_method 0 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7db16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7db16640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7db16640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7d315640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7d315640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7d315640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7db16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7db16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.582 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.582+0000 7f4b7db16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b67fff640 1 -- 10.20.193.189:0/2514659034 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3891604995 0 0) 0x7f4b78122c90 con 0x7f4b78074250 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b67fff640 10 monclient(hunting): _negotiate 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b67fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b67fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b67fff640 10 monclient(hunting): my global_id is 4168 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b67fff640 1 -- 10.20.193.189:0/2514659034 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b60003620 con 0x7f4b78074250 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 10 monclient(hunting): handle_auth_done global_id 4164 payload 274 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 1 -- 10.20.193.189:0/2514659034 >> v2:10.20.193.80:3300/0 conn(0x7f4b78073930 msgr2=0x7f4b78073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 1 --2- 10.20.193.189:0/2514659034 >> v2:10.20.193.80:3300/0 conn(0x7f4b78073930 0x7f4b78073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 1 -- 10.20.193.189:0/2514659034 >> v1:10.20.193.40:6789/0 conn(0x7f4b78074250 legacy=0x7f4b7811e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 10 monclient: _finish_hunting 0 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 1 monclient: found mon.noname-a 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.583+0000 7f4b7db16640 1 -- 10.20.193.189:0/2514659034 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b781241e0 con 0x7f4b781200b0 2026-02-22T23:21:43.583 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.584+0000 7f4b7db16640 1 --2- 10.20.193.189:0/2514659034 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b78122570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b78069850 tx=0x7f4b7402f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51b94ac570fb444 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 1 -- 10.20.193.189:0/2514659034 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7403c070 con 0x7f4b781200b0 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: dump: 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.585 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: _finish_auth 0 2026-02-22T23:21:43.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: _check_auth_tickets 2026-02-22T23:21:43.586 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 1 -- 10.20.193.189:0/2514659034 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7402fe40 con 0x7f4b781200b0 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 1 -- 10.20.193.189:0/2514659034 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74037660 con 0x7f4b781200b0 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b7f5a0640 5 monclient: authenticate success, global_id 4164 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b7f5a0640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b7f5a0640 4 set_mon_vals no callback set 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.585+0000 7f4b67fff640 10 monclient: dump: 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient: shutdown 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2514659034 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 msgr2=0x7f4b78122570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2514659034 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b78122570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b78069850 tx=0x7f4b7402f130 comp rx=0 tx=0).stop 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2514659034 shutdown_connections 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2514659034 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b78122570 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2514659034 >> v2:10.20.193.80:3300/0 conn(0x7f4b78073930 0x7f4b78073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2514659034 >> 10.20.193.189:0/2514659034 conn(0x7f4b7806f640 msgr2=0x7f4b78071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2514659034 shutdown_connections 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2514659034 wait complete. 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient: build_initial_monmap 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient: monmap: 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:43.587 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 librados: starting msgr at 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 librados: starting objecter 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 Processor -- start 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- start start 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 librados: setting wanted keys 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 librados: calling monclient init 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient: init 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4b78073930 0x7f4b781b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): picked mon.noname-c con 0x7f4b78073930 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4b78074250 0x7f4b781b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): picked mon.noname-b con 0x7f4b78074250 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b781ab420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): picked mon.noname-a con 0x7f4b781200b0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78123320 con 0x7f4b781200b0 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781231a0 con 0x7f4b78074250 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781234a0 con 0x7f4b78073930 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.586+0000 7f4b7f5a0640 10 monclient(hunting): authenticate will time out at 981.269767s 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b781ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4b78074250 0x7f4b781b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.588 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4b78073930 0x7f4b781b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b781ab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.189:36980/0 (socket says 10.20.193.189:36980) 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 1 -- 10.20.193.189:0/2299377718 learned_addr learned my addr 10.20.193.189:0/2299377718 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 10 monclient(hunting): get_auth_request con 0x7f4b781200b0 auth_method 0 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 10 monclient(hunting): get_auth_request con 0x7f4b78074250 auth_method 0 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 10 monclient(hunting): get_auth_request con 0x7f4b78073930 auth_method 0 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7db16640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7cb14640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.587+0000 7f4b7d315640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 10 monclient(hunting): handle_auth_done global_id 4167 payload 824 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 1 -- 10.20.193.189:0/2299377718 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4b78073930 msgr2=0x7f4b781b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.589 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4b78073930 0x7f4b781b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 1 -- 10.20.193.189:0/2299377718 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4b78074250 msgr2=0x7f4b781b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4b78074250 0x7f4b781b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 10 monclient: _finish_hunting 0 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 1 monclient: found mon.noname-a 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.588+0000 7f4b7db16640 1 -- 10.20.193.189:0/2299377718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b781abce0 con 0x7f4b781200b0 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.589+0000 7f4b7db16640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b781ab420 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402f660 tx=0x7f4b74004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.589+0000 7f4b65ffb640 10 client.?.objecter ms_handle_connect 0x7f4b781200b0 2026-02-22T23:21:43.590 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.589+0000 7f4b65ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 1 -- 10.20.193.189:0/2299377718 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74047020 con 0x7f4b781200b0 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: dump: 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: _finish_auth 0 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 1 -- 10.20.193.189:0/2299377718 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b74042470 con 0x7f4b781200b0 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 1 -- 10.20.193.189:0/2299377718 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7403c040 con 0x7f4b781200b0 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b65ffb640 10 monclient: dump: 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.591 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b677fe640 4 set_mon_vals no callback set 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 5 monclient: authenticate success, global_id 4167 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 monclient: _renew_subs 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b781abf10 con 0x7f4b781200b0 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 client.4167.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 monclient: _renew_subs 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b781b8120 con 0x7f4b781200b0 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 1 librados: init done 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:43.592 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.591+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b78073d10 con 0x7f4b781200b0 2026-02-22T23:21:43.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.602+0000 7f4b65ffb640 1 -- 10.20.193.189:0/2299377718 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f4b74042860 con 0x7f4b781200b0 2026-02-22T23:21:43.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.602+0000 7f4b65ffb640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4b48078ca0 0x7f4b4807b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.602+0000 7f4b7d315640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4b48078ca0 0x7f4b4807b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b7d315640 10 monclient: get_auth_request con 0x7f4b48078ca0 auth_method 0 2026-02-22T23:21:43.602 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b65ffb640 10 client.4167.objecter ms_handle_connect 0x7f4b48078ca0 2026-02-22T23:21:43.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b7d315640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4b48078ca0 0x7f4b4807b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009c40 tx=0x7f4b68009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b65ffb640 1 -- 10.20.193.189:0/2299377718 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 1990+0+0 (secure 0 0 0) 0x7f4b74083fa0 con 0x7f4b781200b0 2026-02-22T23:21:43.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b65ffb640 10 client.4167.objecter ms_dispatch2 0x7f4b78000b90 osd_map(13..13 src has 1..13) 2026-02-22T23:21:43.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b65ffb640 3 client.4167.objecter handle_osd_map got epochs [13,13] > 0 2026-02-22T23:21:43.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b65ffb640 3 client.4167.objecter handle_osd_map decoding full epoch 13 2026-02-22T23:21:43.603 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.603+0000 7f4b65ffb640 20 client.4167.objecter dump_active .. 0 homeless 2026-02-22T23:21:43.701 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.700+0000 7f4b65ffb640 1 -- 10.20.193.189:0/2299377718 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4b74051030 con 0x7f4b781200b0 2026-02-22T23:21:43.701 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.700+0000 7f4b65ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.701 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.700+0000 7f4b65ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:43.792 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.792+0000 7f4b7f5a0640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "ea741f02-ae0b-419b-8892-34f4b28f6ab6", "id": 11}] 2026-02-22T23:21:43.792 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.792+0000 7f4b7f5a0640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "ea741f02-ae0b-419b-8892-34f4b28f6ab6", "id": 11}] 2026-02-22T23:21:43.792 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.792+0000 7f4b7f5a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:43.793 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.792+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea741f02-ae0b-419b-8892-34f4b28f6ab6", "id": 11} v 0) -- 0x7f4b78119aa0 con 0x7f4b781200b0 2026-02-22T23:21:43.819 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.819+0000 7f4b65ffb640 1 -- 10.20.193.189:0/2299377718 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea741f02-ae0b-419b-8892-34f4b28f6ab6", "id": 11}]=0 v14) ==== 113+0+3 (secure 0 0 0) 0x7f4b78119aa0 con 0x7f4b781200b0 2026-02-22T23:21:43.819 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.819+0000 7f4b65ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "ea741f02-ae0b-419b-8892-34f4b28f6ab6", "id": 11}] 2026-02-22T23:21:43.819 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.819+0000 7f4b65ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:43.819 INFO:teuthology.orchestra.run.trial189.stdout:11 2026-02-22T23:21:43.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 10 librados: watch_flush enter 2026-02-22T23:21:43.821 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 10 librados: watch_flush exit 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 20 client.4167.objecter shutdown clearing up homeless session... 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 10 client.4167.objecter successfully canceled tick 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4b48078ca0 msgr2=0x7f4b4807b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4b48078ca0 0x7f4b4807b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009c40 tx=0x7f4b68009290 comp rx=0 tx=0).stop 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 10 monclient: shutdown 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 msgr2=0x7f4b781ab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b781ab420 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4b7402f660 tx=0x7f4b74004b40 comp rx=0 tx=0).stop 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 shutdown_connections 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4b48078ca0 0x7f4b4807b160 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.822 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4b781200b0 0x7f4b781ab420 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4b78074250 0x7f4b781b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 --2- 10.20.193.189:0/2299377718 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4b78073930 0x7f4b781b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 >> 10.20.193.189:0/2299377718 conn(0x7f4b7806f640 msgr2=0x7f4b78071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:43.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 shutdown_connections 2026-02-22T23:21:43.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.820+0000 7f4b7f5a0640 1 -- 10.20.193.189:0/2299377718 wait complete. 2026-02-22T23:21:43.823 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.821+0000 7f4b7f5a0640 1 librados: shutdown 2026-02-22T23:21:43.828 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 6f306679-2d08-48ea-9fd7-3c4cf4c9323c 12 2026-02-22T23:21:43.897 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient: get_monmap_and_config 2026-02-22T23:21:43.897 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient: build_initial_monmap 2026-02-22T23:21:43.897 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient: monmap: 2026-02-22T23:21:43.897 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:43.897 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.897 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:43.894318+0000 2026-02-22T23:21:43.897 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:43.894318+0000 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 1 Processor -- start 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 1 -- start start 2026-02-22T23:21:43.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient: init 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient: _add_conns ranks=[4,5,3,0,2,1] 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4270073930 0x7f4270073d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient(hunting): picked mon.noname-b con 0x7f4270073930 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4270074250 0x7f427011e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.893+0000 7f4277bcb640 10 monclient(hunting): picked mon.noname-c con 0x7f4270074250 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f427011eed0 0x7f4270121390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 10 monclient(hunting): picked mon.noname-a con 0x7f427011eed0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4270121ab0 con 0x7f427011eed0 2026-02-22T23:21:43.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4270121c30 con 0x7f4270073930 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4270121de0 con 0x7f4270074250 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4277bcb640 10 monclient(hunting): authenticate will time out at 981.577024s 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4276141640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f427011eed0 0x7f4270121390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f427513f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4270074250 0x7f427011e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f427513f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4270074250 0x7f427011e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60426/0 (socket says 10.20.193.189:60426) 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f427513f640 1 -- 10.20.193.189:0/3381490251 learned_addr learned my addr 10.20.193.189:0/3381490251 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f427513f640 10 monclient(hunting): get_auth_request con 0x7f4270074250 auth_method 0 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f427513f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f427513f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f427513f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4276141640 10 monclient(hunting): get_auth_request con 0x7f427011eed0 auth_method 0 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4276141640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4276141640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.894+0000 7f4276141640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f427513f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f427513f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f427513f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4275940640 1 --2- 10.20.193.189:0/3381490251 >> v2:10.20.193.40:3300/0 conn(0x7f4270073930 0x7f4270073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4275940640 10 monclient(hunting): get_auth_request con 0x7f4270073930 auth_method 0 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4275940640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4275940640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4275940640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 10 monclient(hunting): handle_auth_done global_id 4170 payload 274 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 1 -- 10.20.193.189:0/3381490251 >> v2:10.20.193.80:3300/0 conn(0x7f4270074250 msgr2=0x7f427011e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 1 --2- 10.20.193.189:0/3381490251 >> v2:10.20.193.80:3300/0 conn(0x7f4270074250 0x7f427011e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 1 -- 10.20.193.189:0/3381490251 >> v2:10.20.193.40:3300/0 conn(0x7f4270073930 msgr2=0x7f4270073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 1 --2- 10.20.193.189:0/3381490251 >> v2:10.20.193.40:3300/0 conn(0x7f4270073930 0x7f4270073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 10 monclient: _finish_hunting 0 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 1 monclient: found mon.noname-a 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:43.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.895+0000 7f4276141640 1 -- 10.20.193.189:0/3381490251 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4270121f90 con 0x7f427011eed0 2026-02-22T23:21:43.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.896+0000 7f4276141640 1 --2- 10.20.193.189:0/3381490251 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f4270121390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4270069850 tx=0x7f426c02f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366afa40f8fc54e8 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.909 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 1 -- 10.20.193.189:0/3381490251 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c03c070 con 0x7f427011eed0 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: dump: 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.910 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: _finish_auth 0 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: _check_auth_tickets 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 1 -- 10.20.193.189:0/3381490251 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c02fdb0 con 0x7f427011eed0 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 5 monclient: authenticate success, global_id 4170 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 1 -- 10.20.193.189:0/3381490251 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c037660 con 0x7f427011eed0 2026-02-22T23:21:43.911 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f427493e640 10 monclient: dump: 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.912 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 4 set_mon_vals no callback set 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 10 monclient: shutdown 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 -- 10.20.193.189:0/3381490251 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 msgr2=0x7f4270121390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 --2- 10.20.193.189:0/3381490251 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f4270121390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4270069850 tx=0x7f426c02f130 comp rx=0 tx=0).stop 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 -- 10.20.193.189:0/3381490251 shutdown_connections 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 --2- 10.20.193.189:0/3381490251 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f4270121390 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 --2- 10.20.193.189:0/3381490251 >> v2:10.20.193.80:3300/0 conn(0x7f4270074250 0x7f427011e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 --2- 10.20.193.189:0/3381490251 >> v2:10.20.193.40:3300/0 conn(0x7f4270073930 0x7f4270073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 -- 10.20.193.189:0/3381490251 >> 10.20.193.189:0/3381490251 conn(0x7f427006f640 msgr2=0x7f4270071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:43.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 -- 10.20.193.189:0/3381490251 shutdown_connections 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.898+0000 7f4277bcb640 1 -- 10.20.193.189:0/3381490251 wait complete. 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient: build_initial_monmap 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient: monmap: 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:43.914 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 librados: starting msgr at 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 librados: starting objecter 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 Processor -- start 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 -- start start 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 librados: setting wanted keys 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 librados: calling monclient init 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient: init 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4270073930 0x7f42701ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): picked mon.noname-c con 0x7f4270073930 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4270074250 0x7f42701abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): picked mon.noname-b con 0x7f4270074250 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f42701b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): picked mon.noname-a con 0x7f427011eed0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4270122640 con 0x7f427011eed0 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f42701224c0 con 0x7f4270074250 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f42701227c0 con 0x7f4270073930 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4277bcb640 10 monclient(hunting): authenticate will time out at 981.582430s 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4276141640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f42701b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4275940640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4270073930 0x7f42701ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f427513f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4270074250 0x7f42701abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4276141640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f42701b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.189:36992/0 (socket says 10.20.193.189:36992) 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4276141640 1 -- 10.20.193.189:0/2876345650 learned_addr learned my addr 10.20.193.189:0/2876345650 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4276141640 10 monclient(hunting): get_auth_request con 0x7f427011eed0 auth_method 0 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4276141640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4276141640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.899+0000 7f4276141640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f427513f640 10 monclient(hunting): get_auth_request con 0x7f4270074250 auth_method 0 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f427513f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f427513f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f427513f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.917 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4275940640 10 monclient(hunting): get_auth_request con 0x7f4270073930 auth_method 0 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4275940640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4275940640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4275940640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f427513f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f427513f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f427513f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4276141640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4276141640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4276141640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4275940640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4275940640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.900+0000 7f4275940640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:43.918 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 10 monclient(hunting): handle_auth_done global_id 4177 payload 824 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 1 -- 10.20.193.189:0/2876345650 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4270073930 msgr2=0x7f42701ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4270073930 0x7f42701ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 1 -- 10.20.193.189:0/2876345650 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 msgr2=0x7f42701b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f42701b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 10 monclient: _finish_hunting 0 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 1 monclient: found mon.noname-b 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f427513f640 1 -- 10.20.193.189:0/2876345650 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42701b08f0 con 0x7f4270074250 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.901+0000 7f4276141640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f42701b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.902+0000 7f427513f640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4270074250 0x7f42701abfb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4264006dc0 tx=0x7f426400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.902+0000 7f425e7fc640 10 client.?.objecter ms_handle_connect 0x7f4270074250 2026-02-22T23:21:43.919 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.902+0000 7f425e7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f425e7fc640 1 -- 10.20.193.189:0/2876345650 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264017070 con 0x7f4270074250 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f425e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f425e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f425e7fc640 10 monclient: dump: 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.920 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f425e7fc640 10 monclient: _finish_auth 0 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f425e7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f425e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 5 monclient: authenticate success, global_id 4177 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 monclient: _renew_subs 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42701b0be0 con 0x7f4270074250 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 client.4177.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 monclient: _renew_subs 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:43.921 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701b8360 con 0x7f4270074250 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 1 librados: init done 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.903+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270073d10 con 0x7f4270074250 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.904+0000 7f425e7fc640 1 -- 10.20.193.189:0/2876345650 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4264005a90 con 0x7f4270074250 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.904+0000 7f425e7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.904+0000 7f425e7fc640 1 -- 10.20.193.189:0/2876345650 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264021060 con 0x7f4270074250 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.904+0000 7f425e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.904+0000 7f425ffff640 4 set_mon_vals no callback set 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.904+0000 7f425e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:21:43.922 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:43.904+0000 7f425e7fc640 10 monclient: dump: 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:43.923 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.008 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.008+0000 7f425e7fc640 1 -- 10.20.193.189:0/2876345650 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f4264024060 con 0x7f4270074250 2026-02-22T23:21:44.009 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.008+0000 7f425e7fc640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4240078c50 0x7f424007b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.009 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.009+0000 7f4275940640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4240078c50 0x7f424007b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.009 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.009+0000 7f4275940640 10 monclient: get_auth_request con 0x7f4240078c50 auth_method 0 2026-02-22T23:21:44.009 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.009+0000 7f4275940640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4240078c50 0x7f424007b110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4260009a50 tx=0x7f4260008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.010 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.009+0000 7f425e7fc640 10 client.4177.objecter ms_handle_connect 0x7f4240078c50 2026-02-22T23:21:44.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 1 -- 10.20.193.189:0/2876345650 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2105+0+0 (secure 0 0 0) 0x7f426405def0 con 0x7f4270074250 2026-02-22T23:21:44.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 10 client.4177.objecter ms_dispatch2 0x7f4270000b90 osd_map(14..14 src has 1..14) 2026-02-22T23:21:44.015 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 3 client.4177.objecter handle_osd_map got epochs [14,14] > 0 2026-02-22T23:21:44.016 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 3 client.4177.objecter handle_osd_map decoding full epoch 14 2026-02-22T23:21:44.016 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 20 client.4177.objecter dump_active .. 0 homeless 2026-02-22T23:21:44.016 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 1 -- 10.20.193.189:0/2876345650 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4264061eb0 con 0x7f4270074250 2026-02-22T23:21:44.016 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.016 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.015+0000 7f425e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:44.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.102+0000 7f4277bcb640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6f306679-2d08-48ea-9fd7-3c4cf4c9323c", "id": 12}] 2026-02-22T23:21:44.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.102+0000 7f4277bcb640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6f306679-2d08-48ea-9fd7-3c4cf4c9323c", "id": 12}] 2026-02-22T23:21:44.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.102+0000 7f4277bcb640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:21:44.102 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.102+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6f306679-2d08-48ea-9fd7-3c4cf4c9323c", "id": 12} v 0) -- 0x7f42701b86c0 con 0x7f4270074250 2026-02-22T23:21:44.154 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.154+0000 7f425e7fc640 1 -- 10.20.193.189:0/2876345650 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6f306679-2d08-48ea-9fd7-3c4cf4c9323c", "id": 12}]=0 v15) ==== 113+0+3 (secure 0 0 0) 0x7f42640661b0 con 0x7f4270074250 2026-02-22T23:21:44.155 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.154+0000 7f425e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6f306679-2d08-48ea-9fd7-3c4cf4c9323c", "id": 12}] 2026-02-22T23:21:44.155 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.154+0000 7f425e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:44.155 INFO:teuthology.orchestra.run.trial189.stdout:12 2026-02-22T23:21:44.156 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 10 librados: watch_flush enter 2026-02-22T23:21:44.156 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 10 librados: watch_flush exit 2026-02-22T23:21:44.156 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 20 client.4177.objecter shutdown clearing up homeless session... 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 10 client.4177.objecter successfully canceled tick 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4240078c50 msgr2=0x7f424007b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4240078c50 0x7f424007b110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4260009a50 tx=0x7f4260008040 comp rx=0 tx=0).stop 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 10 monclient: shutdown 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4270074250 msgr2=0x7f42701abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4270074250 0x7f42701abfb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4264006dc0 tx=0x7f426400ed70 comp rx=0 tx=0).stop 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 shutdown_connections 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4240078c50 0x7f424007b110 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f427011eed0 0x7f42701b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4270074250 0x7f42701abfb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 --2- 10.20.193.189:0/2876345650 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4270073930 0x7f42701ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 >> 10.20.193.189:0/2876345650 conn(0x7f427006f640 msgr2=0x7f4270071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 shutdown_connections 2026-02-22T23:21:44.157 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 -- 10.20.193.189:0/2876345650 wait complete. 2026-02-22T23:21:44.158 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.155+0000 7f4277bcb640 1 librados: shutdown 2026-02-22T23:21:44.162 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new 6b0643dc-1483-4635-a16d-adf7e477c900 13 2026-02-22T23:21:44.231 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.230+0000 7f2621a57640 10 monclient: get_monmap_and_config 2026-02-22T23:21:44.231 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.230+0000 7f2621a57640 10 monclient: build_initial_monmap 2026-02-22T23:21:44.231 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.230+0000 7f2621a57640 10 monclient: monmap: 2026-02-22T23:21:44.231 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:44.231752+0000 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:44.231752+0000 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 1 Processor -- start 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 1 -- start start 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient: init 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient: _add_conns ranks=[0,2,1,5,3,4] 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f261c073930 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f261c11d960 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f261c121470 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f261c125040 con 0x7f261c073930 2026-02-22T23:21:44.232 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f261c126260 con 0x7f261c121470 2026-02-22T23:21:44.233 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f261c127460 con 0x7f261c11d960 2026-02-22T23:21:44.233 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:44.233 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f2621a57640 10 monclient(hunting): authenticate will time out at 981.914444s 2026-02-22T23:21:44.233 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f261affd640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f261c073930 0x7f261c073d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.189:53868/0 (socket says 10.20.193.189:53868) 2026-02-22T23:21:44.233 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f261affd640 1 -- 10.20.193.189:0/1538564477 learned_addr learned my addr 10.20.193.189:0/1538564477 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:44.233 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.231+0000 7f261b7fe640 1 --1- 10.20.193.189:0/1538564477 >> v1:10.20.193.40:6789/0 conn(0x7f261c121470 0x7f261c123860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.189:59836/0 (socket says 10.20.193.189:59836) 2026-02-22T23:21:44.233 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653531683 0 0) 0x7f261c126260 con 0x7f261c121470 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): my global_id is 4180 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2608003620 con 0x7f261c121470 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132836473 0 0) 0x7f261c125040 con 0x7f261c073930 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): my global_id is 4176 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f261c126260 con 0x7f261c073930 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299364257 0 0) 0x7f261c127460 con 0x7f261c11d960 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 10 monclient(hunting): my global_id is 4205 2026-02-22T23:21:44.234 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.233+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f261c125040 con 0x7f261c11d960 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2807696984 0 0) 0x7f2608003620 con 0x7f261c121470 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 >> v1:10.20.193.80:6789/0 conn(0x7f261c11d960 legacy=0x7f261c11fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 >> v1:10.20.193.6:6789/0 conn(0x7f261c073930 legacy=0x7f261c073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c128660 con 0x7f261c121470 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: _finish_auth 0 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f260c003280 con 0x7f261c121470 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: dump: 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.237 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 >> v1:10.20.193.40:6789/0 conn(0x7f261c121470 legacy=0x7f261c123860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2608003fe0 0x7f2608004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): picked mon.a con 0x7f2608003fe0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2608004e40 0x7f26080052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): picked mon.c con 0x7f2608004e40 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608009ab0 0x7f260800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): picked mon.b con 0x7f2608009ab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f260800c5c0 con 0x7f2608003fe0 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f260800c770 con 0x7f2608009ab0 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f260800c970 con 0x7f2608004e40 2026-02-22T23:21:44.238 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.234+0000 7f2619ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.235+0000 7f261affd640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2608003fe0 0x7f2608004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.235+0000 7f261a7fc640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2608004e40 0x7f26080052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.235+0000 7f261affd640 10 monclient(hunting): get_auth_request con 0x7f2608003fe0 auth_method 0 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.235+0000 7f261affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.235+0000 7f261affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.235+0000 7f261affd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.235+0000 7f261b7fe640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608009ab0 0x7f260800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261a7fc640 10 monclient(hunting): get_auth_request con 0x7f2608004e40 auth_method 0 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261a7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 10 monclient(hunting): get_auth_request con 0x7f2608009ab0 auth_method 0 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.239 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient(hunting): handle_auth_done global_id 4180 payload 293 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 1 -- 10.20.193.189:0/1538564477 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2608004e40 msgr2=0x7f26080052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2608004e40 0x7f26080052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 1 -- 10.20.193.189:0/1538564477 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608009ab0 msgr2=0x7f260800bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608009ab0 0x7f260800bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient: _finish_hunting 0 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 1 monclient: found mon.a 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261b7fe640 1 -- 10.20.193.189:0/1538564477 reap_dead start 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 1 -- 10.20.193.189:0/1538564477 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2608003620 con 0x7f2608003fe0 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient: _finish_auth 0 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f261affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f2621a57640 5 monclient: authenticate success, global_id 4180 2026-02-22T23:21:44.240 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.236+0000 7f2621a57640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.237+0000 7f261affd640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2608003fe0 0x7f2608004900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26100032f0 tx=0x7f261002efe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7875433d534fac15 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610002f10 con 0x7f2608003fe0 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 10 monclient: dump: 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.243 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2610033070 con 0x7f2608003fe0 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 1 -- 10.20.193.189:0/1538564477 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610004de0 con 0x7f2608003fe0 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2619ffb640 10 monclient: dump: 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.244 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 4 set_mon_vals no callback set 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 10 monclient: shutdown 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 1 -- 10.20.193.189:0/1538564477 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2608003fe0 msgr2=0x7f2608004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2608003fe0 0x7f2608004900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f26100032f0 tx=0x7f261002efe0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 1 -- 10.20.193.189:0/1538564477 shutdown_connections 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 1 --2- 10.20.193.189:0/1538564477 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2608003fe0 0x7f2608004900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 1 -- 10.20.193.189:0/1538564477 >> 10.20.193.189:0/1538564477 conn(0x7f261c06f640 msgr2=0x7f261c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 1 -- 10.20.193.189:0/1538564477 shutdown_connections 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.239+0000 7f2621a57640 1 -- 10.20.193.189:0/1538564477 wait complete. 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient: build_initial_monmap 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient: monmap: 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:44.245 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 librados: starting msgr at 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 librados: starting objecter 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 Processor -- start 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 -- start start 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 librados: setting wanted keys 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 librados: calling monclient init 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient: init 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608003fe0 0x7f261c119d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): picked mon.noname-b con 0x7f2608003fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f261c11a2d0 0x7f261c1b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): picked mon.noname-c con 0x7f261c11a2d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f261c1b5a00 0x7f261c1b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): picked mon.noname-a con 0x7f261c1b5a00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f261c074160 con 0x7f261c1b5a00 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f261c073fe0 con 0x7f2608003fe0 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f261c125210 con 0x7f261c11a2d0 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f2621a57640 10 monclient(hunting): authenticate will time out at 981.923427s 2026-02-22T23:21:44.246 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f261c1b5a00 0x7f261c1b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608003fe0 0x7f261c119d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261a7fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f261c11a2d0 0x7f261c1b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f261c1b5a00 0x7f261c1b7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.189:36998/0 (socket says 10.20.193.189:36998) 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261b7fe640 1 -- 10.20.193.189:0/3973289590 learned_addr learned my addr 10.20.193.189:0/3973289590 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261b7fe640 10 monclient(hunting): get_auth_request con 0x7f261c1b5a00 auth_method 0 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.240+0000 7f261b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261affd640 10 monclient(hunting): get_auth_request con 0x7f2608003fe0 auth_method 0 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261a7fc640 10 monclient(hunting): get_auth_request con 0x7f261c11a2d0 auth_method 0 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.247 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.241+0000 7f261a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 10 monclient(hunting): handle_auth_done global_id 4179 payload 824 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 1 -- 10.20.193.189:0/3973289590 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f261c11a2d0 msgr2=0x7f261c1b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f261c11a2d0 0x7f261c1b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 1 -- 10.20.193.189:0/3973289590 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608003fe0 msgr2=0x7f261c119d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608003fe0 0x7f261c119d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.242+0000 7f261b7fe640 1 -- 10.20.193.189:0/3973289590 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c1b83e0 con 0x7f261c1b5a00 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.243+0000 7f261b7fe640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f261c1b5a00 0x7f261c1b7df0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f260c002790 tx=0x7f260c00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.243+0000 7f25f3fff640 10 client.?.objecter ms_handle_connect 0x7f261c1b5a00 2026-02-22T23:21:44.248 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.243+0000 7f25f3fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 1 -- 10.20.193.189:0/3973289590 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c017020 con 0x7f261c1b5a00 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: dump: 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.249 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: _finish_auth 0 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 1 -- 10.20.193.189:0/3973289590 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f260c0133f0 con 0x7f261c1b5a00 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 1 -- 10.20.193.189:0/3973289590 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c00ae60 con 0x7f261c1b5a00 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f26197fa640 4 set_mon_vals no callback set 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f25f3fff640 10 monclient: dump: 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.250 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 5 monclient: authenticate success, global_id 4179 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 10 monclient: _renew_subs 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c1b8670 con 0x7f261c1b5a00 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 10 client.4179.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 10 monclient: _renew_subs 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c1b8bf0 con 0x7f261c1b5a00 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.244+0000 7f2621a57640 1 librados: init done 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.245+0000 7f2621a57640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.245+0000 7f2621a57640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.245+0000 7f2621a57640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.251 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.245+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c1256c0 con 0x7f261c1b5a00 2026-02-22T23:21:44.255 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.255+0000 7f25f3fff640 1 -- 10.20.193.189:0/3973289590 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7f260c0137c0 con 0x7f261c1b5a00 2026-02-22T23:21:44.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.256+0000 7f25f3fff640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f25ec078c50 0x7f25ec07b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.256+0000 7f261affd640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f25ec078c50 0x7f25ec07b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.256+0000 7f261affd640 10 monclient: get_auth_request con 0x7f25ec078c50 auth_method 0 2026-02-22T23:21:44.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.256+0000 7f261affd640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f25ec078c50 0x7f25ec07b110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2610036000 tx=0x7f261003c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.256 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.256+0000 7f25f3fff640 10 client.4179.objecter ms_handle_connect 0x7f25ec078c50 2026-02-22T23:21:44.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.257+0000 7f25f3fff640 1 -- 10.20.193.189:0/3973289590 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2220+0+0 (secure 0 0 0) 0x7f260c05e980 con 0x7f261c1b5a00 2026-02-22T23:21:44.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.257+0000 7f25f3fff640 10 client.4179.objecter ms_dispatch2 0x7f261c000b90 osd_map(15..15 src has 1..15) 2026-02-22T23:21:44.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.257+0000 7f25f3fff640 3 client.4179.objecter handle_osd_map got epochs [15,15] > 0 2026-02-22T23:21:44.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.257+0000 7f25f3fff640 3 client.4179.objecter handle_osd_map decoding full epoch 15 2026-02-22T23:21:44.257 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.257+0000 7f25f3fff640 20 client.4179.objecter dump_active .. 0 homeless 2026-02-22T23:21:44.353 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.353+0000 7f25f3fff640 1 -- 10.20.193.189:0/3973289590 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f260c021020 con 0x7f261c1b5a00 2026-02-22T23:21:44.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.353+0000 7f25f3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.354 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.353+0000 7f25f3fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:44.439 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.439+0000 7f2621a57640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "6b0643dc-1483-4635-a16d-adf7e477c900", "id": 13}] 2026-02-22T23:21:44.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.439+0000 7f2621a57640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "6b0643dc-1483-4635-a16d-adf7e477c900", "id": 13}] 2026-02-22T23:21:44.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.439+0000 7f2621a57640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.440 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.439+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b0643dc-1483-4635-a16d-adf7e477c900", "id": 13} v 0) -- 0x7f261c1229e0 con 0x7f261c1b5a00 2026-02-22T23:21:44.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.479+0000 7f25f3fff640 1 -- 10.20.193.189:0/3973289590 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b0643dc-1483-4635-a16d-adf7e477c900", "id": 13}]=0 v16) ==== 113+0+3 (secure 0 0 0) 0x7f260c063040 con 0x7f261c1b5a00 2026-02-22T23:21:44.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.479+0000 7f25f3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "6b0643dc-1483-4635-a16d-adf7e477c900", "id": 13}] 2026-02-22T23:21:44.479 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.479+0000 7f25f3fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:44.479 INFO:teuthology.orchestra.run.trial189.stdout:13 2026-02-22T23:21:44.480 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 10 librados: watch_flush enter 2026-02-22T23:21:44.480 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 10 librados: watch_flush exit 2026-02-22T23:21:44.480 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 20 client.4179.objecter shutdown clearing up homeless session... 2026-02-22T23:21:44.480 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 10 client.4179.objecter successfully canceled tick 2026-02-22T23:21:44.480 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f25ec078c50 msgr2=0x7f25ec07b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.480 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f25ec078c50 0x7f25ec07b110 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2610036000 tx=0x7f261003c040 comp rx=0 tx=0).stop 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 10 monclient: shutdown 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f261c1b5a00 msgr2=0x7f261c1b7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f261c1b5a00 0x7f261c1b7df0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f260c002790 tx=0x7f260c00a640 comp rx=0 tx=0).stop 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 shutdown_connections 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f25ec078c50 0x7f25ec07b110 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f261c1b5a00 0x7f261c1b7df0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f261c11a2d0 0x7f261c1b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 --2- 10.20.193.189:0/3973289590 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2608003fe0 0x7f261c119d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 >> 10.20.193.189:0/3973289590 conn(0x7f261c06f640 msgr2=0x7f261c11fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 shutdown_connections 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 -- 10.20.193.189:0/3973289590 wait complete. 2026-02-22T23:21:44.481 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.480+0000 7f2621a57640 1 librados: shutdown 2026-02-22T23:21:44.487 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new cce82a9e-9fa7-4718-ad88-757e1e46acde 14 2026-02-22T23:21:44.553 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.551+0000 7fa0bc386640 10 monclient: get_monmap_and_config 2026-02-22T23:21:44.553 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.551+0000 7fa0bc386640 10 monclient: build_initial_monmap 2026-02-22T23:21:44.553 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.551+0000 7fa0bc386640 10 monclient: monmap: 2026-02-22T23:21:44.553 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:44.553 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.553 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:44.552458+0000 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:44.552458+0000 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.551+0000 7fa0bc386640 1 Processor -- start 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.551+0000 7fa0bc386640 1 -- start start 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.551+0000 7fa0bc386640 10 monclient: init 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient: _add_conns ranks=[1,5,2,0,3,4] 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa0b4073930 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa0b4117280 0x7fa0b4117710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient(hunting): picked mon.noname-c con 0x7fa0b4117280 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa0b4117c50 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b4128400 con 0x7fa0b4073930 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b41295e0 con 0x7fa0b4117c50 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b412a7c0 con 0x7fa0b4117280 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0bc386640 10 monclient(hunting): authenticate will time out at 982.235153s 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0ba0fb640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fa0b4073930 0x7fa0b4073d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.189:59852/0 (socket says 10.20.193.189:59852) 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0ba0fb640 1 -- 10.20.193.189:0/3802995636 learned_addr learned my addr 10.20.193.189:0/3802995636 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.552+0000 7fa0b98fa640 1 --2- 10.20.193.189:0/3802995636 >> v2:10.20.193.80:3300/0 conn(0x7fa0b4117280 0x7fa0b4117710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.553+0000 7fa0b98fa640 10 monclient(hunting): get_auth_request con 0x7fa0b4117280 auth_method 0 2026-02-22T23:21:44.554 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.553+0000 7fa0b98fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.553+0000 7fa0b98fa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.553+0000 7fa0b98fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.553+0000 7fa0b98fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.553+0000 7fa0b98fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.553+0000 7fa0b98fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 10 monclient(hunting): handle_auth_done global_id 4211 payload 274 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 1 -- 10.20.193.189:0/3802995636 >> v1:10.20.193.80:6789/0 conn(0x7fa0b4117c50 legacy=0x7fa0b4118100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 1 -- 10.20.193.189:0/3802995636 >> v1:10.20.193.40:6789/0 conn(0x7fa0b4073930 legacy=0x7fa0b4073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 10 monclient: _finish_hunting 0 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 1 monclient: found mon.noname-c 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:44.555 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.554+0000 7fa0b98fa640 1 -- 10.20.193.189:0/3802995636 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b412a940 con 0x7fa0b4117280 2026-02-22T23:21:44.556 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.556+0000 7fa0b98fa640 1 --2- 10.20.193.189:0/3802995636 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117280 0x7fa0b4117710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098a0 tx=0x7fa0a802edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6557a08722b6ec5 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 1 -- 10.20.193.189:0/3802995636 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a803c070 con 0x7fa0b4117280 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: dump: 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: _finish_auth 0 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: _check_auth_tickets 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 1 -- 10.20.193.189:0/3802995636 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a802fae0 con 0x7fa0b4117280 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0bc386640 5 monclient: authenticate success, global_id 4211 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0bc386640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 1 -- 10.20.193.189:0/3802995636 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8037440 con 0x7fa0b4117280 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.558 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0b90f9640 10 monclient: dump: 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0bc386640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.557+0000 7fa0bc386640 4 set_mon_vals no callback set 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 10 monclient: shutdown 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 -- 10.20.193.189:0/3802995636 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117280 msgr2=0x7fa0b4117710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3802995636 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117280 0x7fa0b4117710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098a0 tx=0x7fa0a802edd0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 -- 10.20.193.189:0/3802995636 shutdown_connections 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3802995636 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117280 0x7fa0b4117710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 -- 10.20.193.189:0/3802995636 >> 10.20.193.189:0/3802995636 conn(0x7fa0b406f640 msgr2=0x7fa0b4071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 -- 10.20.193.189:0/3802995636 shutdown_connections 2026-02-22T23:21:44.559 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 -- 10.20.193.189:0/3802995636 wait complete. 2026-02-22T23:21:44.560 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 10 monclient: build_initial_monmap 2026-02-22T23:21:44.560 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 10 monclient: monmap: 2026-02-22T23:21:44.560 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:44.560 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:44.560 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:44.560 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:44.560 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 librados: starting msgr at 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 librados: starting objecter 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 Processor -- start 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 -- start start 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 librados: setting wanted keys 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 1 librados: calling monclient init 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.558+0000 7fa0bc386640 10 monclient: init 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa0b4073930 0x7fa0b41b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): picked mon.noname-a con 0x7fa0b4073930 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa0b4117280 0x7fa0b41b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): picked mon.noname-b con 0x7fa0b4117280 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117c50 0x7fa0b4126960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): picked mon.noname-c con 0x7fa0b4117c50 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4129c30 con 0x7fa0b4073930 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4129ab0 con 0x7fa0b4117280 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4129db0 con 0x7fa0b4117c50 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0bc386640 10 monclient(hunting): authenticate will time out at 982.242143s 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0b98fa640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa0b4117280 0x7fa0b41b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117c50 0x7fa0b4126960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.561 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117c50 0x7fa0b4126960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.189:60478/0 (socket says 10.20.193.189:60478) 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 1 -- 10.20.193.189:0/3197707587 learned_addr learned my addr 10.20.193.189:0/3197707587 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 10 monclient(hunting): get_auth_request con 0x7fa0b4117c50 auth_method 0 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0b98fa640 10 monclient(hunting): get_auth_request con 0x7fa0b4117280 auth_method 0 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0b98fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0b98fa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0b98fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.559+0000 7fa0ba8fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.560+0000 7fa0b98fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.560+0000 7fa0b98fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.560+0000 7fa0b98fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.562 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 10 monclient(hunting): handle_auth_done global_id 4214 payload 824 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 1 -- 10.20.193.189:0/3197707587 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa0b4117280 msgr2=0x7fa0b41b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa0b4117280 0x7fa0b41b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 1 -- 10.20.193.189:0/3197707587 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa0b4073930 msgr2=0x7fa0b41b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa0b4073930 0x7fa0b41b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 10 monclient: _finish_hunting 0 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 1 monclient: found mon.noname-c 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.561+0000 7fa0ba8fc640 1 -- 10.20.193.189:0/3197707587 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b4127220 con 0x7fa0b4117c50 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.562+0000 7fa0ba8fc640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117c50 0x7fa0b4126960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000cc50 tx=0x7fa0b000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.562+0000 7fa09affd640 10 client.?.objecter ms_handle_connect 0x7fa0b4117c50 2026-02-22T23:21:44.563 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.562+0000 7fa09affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003a70 con 0x7fa0b4117c50 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: dump: 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: _finish_auth 0 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b0003c10 con 0x7fa0b4117c50 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b00050d0 con 0x7fa0b4117c50 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:44.564 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa09affd640 10 monclient: dump: 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0b88f8640 4 set_mon_vals no callback set 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 5 monclient: authenticate success, global_id 4214 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 10 monclient: _renew_subs 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b4127510 con 0x7fa0b4117c50 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 10 client.4214.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 10 monclient: _renew_subs 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41bcc40 con 0x7fa0b4117c50 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.563+0000 7fa0bc386640 1 librados: init done 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.564+0000 7fa0bc386640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.564+0000 7fa0bc386640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.564+0000 7fa0bc386640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:44.565 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.564+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4117790 con 0x7fa0b4117c50 2026-02-22T23:21:44.574 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 3) ==== 99739+0+0 (secure 0 0 0) 0x7fa0b0005270 con 0x7fa0b4117c50 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa084078ca0 0x7fa08407b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa0ba0fb640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa084078ca0 0x7fa08407b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa0ba0fb640 10 monclient: get_auth_request con 0x7fa084078ca0 auth_method 0 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2335+0+0 (secure 0 0 0) 0x7fa0b005e6d0 con 0x7fa0b4117c50 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 10 client.4214.objecter ms_dispatch2 0x7fa0b4000b90 osd_map(16..16 src has 1..16) 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 3 client.4214.objecter handle_osd_map got epochs [16,16] > 0 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 3 client.4214.objecter handle_osd_map decoding full epoch 16 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 20 client.4214.objecter dump_active .. 0 homeless 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa09affd640 10 client.4214.objecter ms_handle_connect 0x7fa084078ca0 2026-02-22T23:21:44.575 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.575+0000 7fa0ba0fb640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa084078ca0 0x7fa08407b160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4007fd0 tx=0x7fa0a4007b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.671+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa0b0062d90 con 0x7fa0b4117c50 2026-02-22T23:21:44.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.671+0000 7fa09affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.671 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.671+0000 7fa09affd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:44.728 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.727+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7fa0b0061f50 con 0x7fa0b4117c50 2026-02-22T23:21:44.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.759+0000 7fa0bc386640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "cce82a9e-9fa7-4718-ad88-757e1e46acde", "id": 14}] 2026-02-22T23:21:44.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.759+0000 7fa0bc386640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "cce82a9e-9fa7-4718-ad88-757e1e46acde", "id": 14}] 2026-02-22T23:21:44.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.759+0000 7fa0bc386640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:44.760 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.759+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cce82a9e-9fa7-4718-ad88-757e1e46acde", "id": 14} v 0) -- 0x7fa0b4116320 con 0x7fa0b4117c50 2026-02-22T23:21:44.809 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.809+0000 7fa09affd640 1 -- 10.20.193.189:0/3197707587 <== mon.2 v2:10.20.193.80:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cce82a9e-9fa7-4718-ad88-757e1e46acde", "id": 14}]=0 v17) ==== 113+0+3 (secure 0 0 0) 0x7fa0b0012420 con 0x7fa0b4117c50 2026-02-22T23:21:44.809 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.809+0000 7fa09affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "cce82a9e-9fa7-4718-ad88-757e1e46acde", "id": 14}] 2026-02-22T23:21:44.809 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.809+0000 7fa09affd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:44.810 INFO:teuthology.orchestra.run.trial189.stdout:14 2026-02-22T23:21:44.811 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 10 librados: watch_flush enter 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 10 librados: watch_flush exit 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 20 client.4214.objecter shutdown clearing up homeless session... 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 10 client.4214.objecter successfully canceled tick 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa084078ca0 msgr2=0x7fa08407b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa084078ca0 0x7fa08407b160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4007fd0 tx=0x7fa0a4007b20 comp rx=0 tx=0).stop 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 10 monclient: shutdown 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117c50 msgr2=0x7fa0b4126960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117c50 0x7fa0b4126960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000cc50 tx=0x7fa0b000a430 comp rx=0 tx=0).stop 2026-02-22T23:21:44.812 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 shutdown_connections 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa084078ca0 0x7fa08407b160 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa0b4117c50 0x7fa0b4126960 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa0b4117280 0x7fa0b41b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 --2- 10.20.193.189:0/3197707587 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa0b4073930 0x7fa0b41b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 >> 10.20.193.189:0/3197707587 conn(0x7fa0b406f640 msgr2=0x7fa0b40707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 shutdown_connections 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 -- 10.20.193.189:0/3197707587 wait complete. 2026-02-22T23:21:44.813 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.810+0000 7fa0bc386640 1 librados: shutdown 2026-02-22T23:21:44.820 DEBUG:teuthology.orchestra.run.trial189:> sudo ceph --cluster ceph osd new aec0b3b0-8781-48f2-9d8d-a779e70b174f 15 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient: get_monmap_and_config 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient: build_initial_monmap 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient: monmap: 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:21:44.887183+0000 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:21:44.887183+0000 2026-02-22T23:21:44.888 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 1 Processor -- start 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 1 -- start start 2026-02-22T23:21:44.889 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient: init 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient: _add_conns ranks=[2,0,3,5,1,4] 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3544073930 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3544117280 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f3544117e20 0x7f35441182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient(hunting): picked mon.noname-a con 0x7f3544117e20 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3544128400 con 0x7f3544117280 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35441295e0 con 0x7f3544073930 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f354412a7c0 con 0x7f3544117e20 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:44.890 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.886+0000 7f35492ea640 10 monclient(hunting): authenticate will time out at 982.569881s 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.887+0000 7f35437fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f3544117e20 0x7f35441182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.887+0000 7f3542ffd640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f3544073930 0x7f3544073d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.189:56954/0 (socket says 10.20.193.189:56954) 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.887+0000 7f3542ffd640 1 -- 10.20.193.189:0/1135892566 learned_addr learned my addr 10.20.193.189:0/1135892566 (peer_addr_for_me v1:10.20.193.189:0/0) 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.887+0000 7f35437fe640 10 monclient(hunting): get_auth_request con 0x7f3544117e20 auth_method 0 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.887+0000 7f35437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.887+0000 7f35437fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.887+0000 7f35437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.891 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f3541ffb640 1 -- 10.20.193.189:0/1135892566 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351848722 0 0) 0x7f35441295e0 con 0x7f3544073930 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f3541ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f3541ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f3541ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f3541ffb640 10 monclient(hunting): my global_id is 4217 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f3541ffb640 1 -- 10.20.193.189:0/1135892566 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3530003620 con 0x7f3544073930 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 10 monclient(hunting): handle_auth_done global_id 4182 payload 274 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 1 -- 10.20.193.189:0/1135892566 >> v1:10.20.193.80:6789/0 conn(0x7f3544073930 legacy=0x7f3544073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 1 -- 10.20.193.189:0/1135892566 >> v1:10.20.193.6:6789/0 conn(0x7f3544117280 legacy=0x7f3544117710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.898 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 10 monclient: _finish_hunting 0 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 1 monclient: found mon.noname-a 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.888+0000 7f35437fe640 1 -- 10.20.193.189:0/1135892566 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354412a940 con 0x7f3544117e20 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.891+0000 7f35437fe640 1 --2- 10.20.193.189:0/1135892566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117e20 0x7f35441182d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3534009d90 tx=0x7f353402f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ee3cf5c2110e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 1 -- 10.20.193.189:0/1135892566 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353403c070 con 0x7f3544117e20 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: dump: 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.899 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: _finish_auth 0 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:44.900 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 1 -- 10.20.193.189:0/1135892566 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3534037440 con 0x7f3544117e20 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f35492ea640 5 monclient: authenticate success, global_id 4182 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f35492ea640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f35492ea640 4 set_mon_vals no callback set 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 1 -- 10.20.193.189:0/1135892566 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534037810 con 0x7f3544117e20 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.892+0000 7f3541ffb640 10 monclient: dump: 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.901 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient: shutdown 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- 10.20.193.189:0/1135892566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117e20 msgr2=0x7f35441182d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 --2- 10.20.193.189:0/1135892566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117e20 0x7f35441182d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3534009d90 tx=0x7f353402f1e0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- 10.20.193.189:0/1135892566 shutdown_connections 2026-02-22T23:21:44.902 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 --2- 10.20.193.189:0/1135892566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117e20 0x7f35441182d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- 10.20.193.189:0/1135892566 >> 10.20.193.189:0/1135892566 conn(0x7f354406f640 msgr2=0x7f3544071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- 10.20.193.189:0/1135892566 shutdown_connections 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- 10.20.193.189:0/1135892566 wait complete. 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient: build_initial_monmap 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient: monmap: 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:epoch 0 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 0.000000 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:created 0.000000 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:44.903 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 librados: starting msgr at 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 librados: starting objecter 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 Processor -- start 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- start start 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 librados: setting wanted keys 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 librados: calling monclient init 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient: init 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:21:44.904 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3544073930 0x7f35441268d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): picked mon.noname-c con 0x7f3544073930 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117280 0x7f3544126f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): picked mon.noname-a con 0x7f3544117280 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3544117e20 0x7f35441b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): picked mon.noname-b con 0x7f3544117e20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3544128a50 con 0x7f3544117280 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f35441288d0 con 0x7f3544117e20 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3544128bd0 con 0x7f3544073930 2026-02-22T23:21:44.905 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.893+0000 7f35492ea640 10 monclient(hunting): authenticate will time out at 982.576753s 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117280 0x7f3544126f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3544117e20 0x7f35441b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3544073930 0x7f35441268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117280 0x7f3544126f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.189:37028/0 (socket says 10.20.193.189:37028) 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 1 -- 10.20.193.189:0/2146870552 learned_addr learned my addr 10.20.193.189:0/2146870552 (peer_addr_for_me v2:10.20.193.189:0/0) 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 10 monclient(hunting): get_auth_request con 0x7f3544117280 auth_method 0 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 10 monclient(hunting): get_auth_request con 0x7f3544073930 auth_method 0 2026-02-22T23:21:44.906 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 10 monclient(hunting): get_auth_request con 0x7f3544117e20 auth_method 0 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35427fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.907 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f3542ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.894+0000 7f35437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 10 monclient(hunting): handle_auth_done global_id 4188 payload 824 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 1 -- 10.20.193.189:0/2146870552 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3544073930 msgr2=0x7f35441268d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3544073930 0x7f35441268d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 1 -- 10.20.193.189:0/2146870552 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3544117e20 msgr2=0x7f35441b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3544117e20 0x7f35441b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 10 monclient: _finish_hunting 0 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 1 monclient: found mon.noname-a 2026-02-22T23:21:44.908 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:44.909 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.909 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.895+0000 7f35427fc640 1 -- 10.20.193.189:0/2146870552 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35441b6060 con 0x7f3544117280 2026-02-22T23:21:44.909 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.896+0000 7f35427fc640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117280 0x7f3544126f40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f352c00c700 tx=0x7f352c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.909 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.896+0000 7f35237fe640 10 client.?.objecter ms_handle_connect 0x7f3544117280 2026-02-22T23:21:44.909 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.896+0000 7f35237fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 1 -- 10.20.193.189:0/2146870552 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0052d0 con 0x7f3544117280 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: dump: 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.912 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: _finish_auth 0 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: _check_auth_tickets 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 1 -- 10.20.193.189:0/2146870552 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c005470 con 0x7f3544117280 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 1 -- 10.20.193.189:0/2146870552 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0038d0 con 0x7f3544117280 2026-02-22T23:21:44.913 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35417fa640 4 set_mon_vals no callback set 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35237fe640 10 monclient: dump: 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:epoch 1 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:election_strategy: 1 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:44.914 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 5 monclient: authenticate success, global_id 4188 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 10 monclient: _renew_subs 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35441b62f0 con 0x7f3544117280 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 10 client.4188.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 10 monclient: _renew_subs 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35441bc9e0 con 0x7f3544117280 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.898+0000 7f35492ea640 1 librados: init done 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.899+0000 7f35492ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.899+0000 7f35492ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:44.915 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.899+0000 7f35492ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.899+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3544118350 con 0x7f3544117280 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.909+0000 7f35237fe640 1 -- 10.20.193.189:0/2146870552 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f352c003a70 con 0x7f3544117280 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.910+0000 7f35237fe640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3510078d70 0x7f351007b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.910+0000 7f3542ffd640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3510078d70 0x7f351007b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.910+0000 7f3542ffd640 10 monclient: get_auth_request con 0x7f3510078d70 auth_method 0 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.910+0000 7f3542ffd640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3510078d70 0x7f351007b230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3538005980 tx=0x7f3538007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.910+0000 7f35237fe640 10 client.4188.objecter ms_handle_connect 0x7f3510078d70 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.911+0000 7f35237fe640 1 -- 10.20.193.189:0/2146870552 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2450+0+0 (secure 0 0 0) 0x7f352c05e040 con 0x7f3544117280 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.911+0000 7f35237fe640 10 client.4188.objecter ms_dispatch2 0x7f3544000b90 osd_map(17..17 src has 1..17) 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.911+0000 7f35237fe640 3 client.4188.objecter handle_osd_map got epochs [17,17] > 0 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.911+0000 7f35237fe640 3 client.4188.objecter handle_osd_map decoding full epoch 17 2026-02-22T23:21:44.916 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:44.911+0000 7f35237fe640 20 client.4188.objecter dump_active .. 0 homeless 2026-02-22T23:21:45.011 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.010+0000 7f35237fe640 1 -- 10.20.193.189:0/2146870552 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f352c062000 con 0x7f3544117280 2026-02-22T23:21:45.011 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.010+0000 7f35237fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:45.011 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.010+0000 7f35237fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:45.099 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.098+0000 7f35492ea640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "aec0b3b0-8781-48f2-9d8d-a779e70b174f", "id": 15}] 2026-02-22T23:21:45.099 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.098+0000 7f35492ea640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "aec0b3b0-8781-48f2-9d8d-a779e70b174f", "id": 15}] 2026-02-22T23:21:45.099 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.098+0000 7f35492ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:45.099 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.098+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aec0b3b0-8781-48f2-9d8d-a779e70b174f", "id": 15} v 0) -- 0x7f35441bcbb0 con 0x7f3544117280 2026-02-22T23:21:45.115 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.115+0000 7f35237fe640 1 -- 10.20.193.189:0/2146870552 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aec0b3b0-8781-48f2-9d8d-a779e70b174f", "id": 15}]=0 v18) ==== 113+0+3 (secure 0 0 0) 0x7f352c066300 con 0x7f3544117280 2026-02-22T23:21:45.115 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.115+0000 7f35237fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "aec0b3b0-8781-48f2-9d8d-a779e70b174f", "id": 15}] 2026-02-22T23:21:45.116 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.115+0000 7f35237fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:45.116 INFO:teuthology.orchestra.run.trial189.stdout:15 2026-02-22T23:21:45.117 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 10 librados: watch_flush enter 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 10 librados: watch_flush exit 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 20 client.4188.objecter shutdown clearing up homeless session... 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 10 client.4188.objecter successfully canceled tick 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3510078d70 msgr2=0x7f351007b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3510078d70 0x7f351007b230 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3538005980 tx=0x7f3538007dd0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 10 monclient: shutdown 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117280 msgr2=0x7f3544126f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117280 0x7f3544126f40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f352c00c700 tx=0x7f352c00cbd0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 shutdown_connections 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3510078d70 0x7f351007b230 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.118 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3544117e20 0x7f35441b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3544117280 0x7f3544126f40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 --2- 10.20.193.189:0/2146870552 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3544073930 0x7f35441268d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 >> 10.20.193.189:0/2146870552 conn(0x7f354406f640 msgr2=0x7f35440704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:45.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 shutdown_connections 2026-02-22T23:21:45.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 -- 10.20.193.189:0/2146870552 wait complete. 2026-02-22T23:21:45.119 INFO:teuthology.orchestra.run.trial189.stderr:2026-02-22T23:21:45.116+0000 7f35492ea640 1 librados: shutdown 2026-02-22T23:21:45.124 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.124 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-22T23:21:45.125 DEBUG:teuthology.orchestra.run.trial006:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2026-02-22T23:21:45.127 INFO:tasks.ceph.osd.0:Started 2026-02-22T23:21:45.127 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.127 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-22T23:21:45.128 DEBUG:teuthology.orchestra.run.trial006:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2026-02-22T23:21:45.130 INFO:tasks.ceph.osd.4:Started 2026-02-22T23:21:45.130 DEBUG:tasks.ceph_manager:running osd.8 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.8.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.130 INFO:tasks.ceph.osd.8:Restarting daemon 2026-02-22T23:21:45.130 DEBUG:teuthology.orchestra.run.trial006:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 2026-02-22T23:21:45.133 INFO:tasks.ceph.osd.8:Started 2026-02-22T23:21:45.133 DEBUG:tasks.ceph_manager:running osd.12 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.12.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.134 INFO:tasks.ceph.osd.12:Restarting daemon 2026-02-22T23:21:45.134 DEBUG:teuthology.orchestra.run.trial006:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 2026-02-22T23:21:45.136 INFO:tasks.ceph.osd.12:Started 2026-02-22T23:21:45.136 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.137 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-22T23:21:45.137 DEBUG:teuthology.orchestra.run.trial040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2026-02-22T23:21:45.140 INFO:tasks.ceph.osd.1:Started 2026-02-22T23:21:45.140 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.140 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-22T23:21:45.141 DEBUG:teuthology.orchestra.run.trial040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2026-02-22T23:21:45.143 INFO:tasks.ceph.osd.5:Started 2026-02-22T23:21:45.144 DEBUG:tasks.ceph_manager:running osd.9 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.9.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.144 INFO:tasks.ceph.osd.9:Restarting daemon 2026-02-22T23:21:45.144 DEBUG:teuthology.orchestra.run.trial040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 2026-02-22T23:21:45.146 INFO:tasks.ceph.osd.9:Started 2026-02-22T23:21:45.146 DEBUG:tasks.ceph_manager:running osd.13 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.13.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.147 INFO:tasks.ceph.osd.13:Restarting daemon 2026-02-22T23:21:45.147 DEBUG:teuthology.orchestra.run.trial040:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 2026-02-22T23:21:45.206 INFO:tasks.ceph.osd.13:Started 2026-02-22T23:21:45.206 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.207 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-22T23:21:45.207 DEBUG:teuthology.orchestra.run.trial080:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2026-02-22T23:21:45.209 INFO:tasks.ceph.osd.2:Started 2026-02-22T23:21:45.209 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.209 INFO:tasks.ceph.osd.6:Restarting daemon 2026-02-22T23:21:45.210 DEBUG:teuthology.orchestra.run.trial080:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2026-02-22T23:21:45.211 INFO:tasks.ceph.osd.6:Started 2026-02-22T23:21:45.212 DEBUG:tasks.ceph_manager:running osd.10 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.10.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.212 INFO:tasks.ceph.osd.10:Restarting daemon 2026-02-22T23:21:45.212 DEBUG:teuthology.orchestra.run.trial080:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 2026-02-22T23:21:45.214 INFO:tasks.ceph.osd.10:Started 2026-02-22T23:21:45.214 DEBUG:tasks.ceph_manager:running osd.14 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.14.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.214 INFO:tasks.ceph.osd.14:Restarting daemon 2026-02-22T23:21:45.214 DEBUG:teuthology.orchestra.run.trial080:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 2026-02-22T23:21:45.216 INFO:tasks.ceph.osd.14:Started 2026-02-22T23:21:45.216 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.216 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-22T23:21:45.216 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2026-02-22T23:21:45.218 INFO:tasks.ceph.osd.3:Started 2026-02-22T23:21:45.219 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.219 INFO:tasks.ceph.osd.7:Restarting daemon 2026-02-22T23:21:45.219 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2026-02-22T23:21:45.221 INFO:tasks.ceph.osd.7:Started 2026-02-22T23:21:45.221 DEBUG:tasks.ceph_manager:running osd.11 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.11.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.221 INFO:tasks.ceph.osd.11:Restarting daemon 2026-02-22T23:21:45.221 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 2026-02-22T23:21:45.223 INFO:tasks.ceph.osd.11:Started 2026-02-22T23:21:45.223 DEBUG:tasks.ceph_manager:running osd.15 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.15.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-22T23:21:45.223 INFO:tasks.ceph.osd.15:Restarting daemon 2026-02-22T23:21:45.223 DEBUG:teuthology.orchestra.run.trial189:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 2026-02-22T23:21:45.225 INFO:tasks.ceph.osd.15:Started 2026-02-22T23:21:45.226 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.292+0000 7f297c2a0640 10 monclient: get_monmap_and_config 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.292+0000 7f297c2a0640 10 monclient: build_initial_monmap 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.292+0000 7f297c2a0640 10 monclient: monmap: 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:21:45.293164+0000 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:21:45.293164+0000 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:45.294 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:45.295 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:45.295 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:45.295 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:45.295 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:45.295 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:45.295 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:45.295 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 1 Processor -- start 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 1 -- start start 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient: init 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient: _add_conns ranks=[3,1,4,2,5,0] 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f29741527f0 0x7f2974073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient(hunting): picked mon.noname-a con 0x7f29741527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f297415d2c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f297415d730 0x7f29740ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:45.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient(hunting): picked mon.noname-b con 0x7f297415d730 addr v2:10.20.193.40:3300/0 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2974073c20 con 0x7f297415d2c0 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2974074b10 con 0x7f29741527f0 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2974073f00 con 0x7f297415d730 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.293+0000 7f297c2a0640 10 monclient(hunting): authenticate will time out at 980.020734s 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f29741527f0 0x7f2974073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f29741527f0 0x7f2974073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:46854/0 (socket says 10.20.193.6:46854) 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 1 -- 10.20.193.6:0/4083133107 learned_addr learned my addr 10.20.193.6:0/4083133107 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 10 monclient(hunting): get_auth_request con 0x7f29741527f0 auth_method 0 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a816640 1 --2- 10.20.193.6:0/4083133107 >> v2:10.20.193.40:3300/0 conn(0x7f297415d730 0x7f29740ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:45.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.294+0000 7f297a015640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a816640 10 monclient(hunting): get_auth_request con 0x7f297415d730 auth_method 0 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a816640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a816640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 10 monclient(hunting): handle_auth_done global_id 4191 payload 274 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 1 -- 10.20.193.6:0/4083133107 >> v2:10.20.193.40:3300/0 conn(0x7f297415d730 msgr2=0x7f29740ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 1 --2- 10.20.193.6:0/4083133107 >> v2:10.20.193.40:3300/0 conn(0x7f297415d730 0x7f29740ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 1 -- 10.20.193.6:0/4083133107 >> v1:10.20.193.40:6789/0 conn(0x7f297415d2c0 legacy=0x7f297415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 10 monclient: _finish_hunting 0 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 1 monclient: found mon.noname-a 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:45.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.295+0000 7f297a015640 1 -- 10.20.193.6:0/4083133107 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29740741e0 con 0x7f29741527f0 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.296+0000 7f297a015640 1 --2- 10.20.193.6:0/4083133107 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f2974073530 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f297406cdf0 tx=0x7f296402eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ee04752f18e4c1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 1 -- 10.20.193.6:0/4083133107 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296403c070 con 0x7f29741527f0 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: dump: 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:45.301 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: _finish_auth 0 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: _check_auth_tickets 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 1 -- 10.20.193.6:0/4083133107 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296402fbe0 con 0x7f29741527f0 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 1 -- 10.20.193.6:0/4083133107 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964037440 con 0x7f29741527f0 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f2979013640 10 monclient: dump: 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:45.302 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f297c2a0640 5 monclient: authenticate success, global_id 4191 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f297c2a0640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.298+0000 7f297c2a0640 4 set_mon_vals no callback set 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 10 monclient: shutdown 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 -- 10.20.193.6:0/4083133107 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 msgr2=0x7f2974073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 --2- 10.20.193.6:0/4083133107 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f2974073530 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f297406cdf0 tx=0x7f296402eed0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 -- 10.20.193.6:0/4083133107 shutdown_connections 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 --2- 10.20.193.6:0/4083133107 >> v2:10.20.193.40:3300/0 conn(0x7f297415d730 0x7f29740ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 --2- 10.20.193.6:0/4083133107 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f2974073530 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 -- 10.20.193.6:0/4083133107 >> 10.20.193.6:0/4083133107 conn(0x7f29740ce730 msgr2=0x7f29740ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 -- 10.20.193.6:0/4083133107 shutdown_connections 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 -- 10.20.193.6:0/4083133107 wait complete. 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 10 monclient: build_initial_monmap 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 10 monclient: monmap: 2026-02-22T23:21:45.303 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 librados: starting msgr at 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 librados: starting objecter 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 Processor -- start 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 -- start start 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 librados: setting wanted keys 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 1 librados: calling monclient init 2026-02-22T23:21:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.299+0000 7f297c2a0640 10 monclient: init 2026-02-22T23:21:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:21:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f29741f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): picked mon.noname-a con 0x7f29741527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f297415d2c0 0x7f29741f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): picked mon.noname-c con 0x7f297415d2c0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f297415d730 0x7f29740ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): picked mon.noname-b con 0x7f297415d730 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2974074fa0 con 0x7f29741527f0 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f29740747a0 con 0x7f297415d730 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2974075120 con 0x7f297415d2c0 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297c2a0640 10 monclient(hunting): authenticate will time out at 980.027483s 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a015640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f29741f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a015640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f29741f32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:46864/0 (socket says 10.20.193.6:46864) 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a015640 1 -- 10.20.193.6:0/2589765919 learned_addr learned my addr 10.20.193.6:0/2589765919 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:21:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a816640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f297415d730 0x7f29740ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a015640 10 monclient(hunting): get_auth_request con 0x7f29741527f0 auth_method 0 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a015640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a015640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.300+0000 7f297a015640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a816640 10 monclient(hunting): get_auth_request con 0x7f297415d730 auth_method 0 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a816640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a816640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a816640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f297415d2c0 0x7f29741f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a015640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a015640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a015640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 10 monclient(hunting): get_auth_request con 0x7f297415d2c0 auth_method 0 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a816640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a816640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f297a816640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.301+0000 7f2979814640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.302+0000 7f297a015640 10 monclient(hunting): handle_auth_done global_id 4194 payload 824 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 1 -- 10.20.193.6:0/2589765919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f297415d2c0 msgr2=0x7f29741f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f297415d2c0 0x7f29741f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 1 -- 10.20.193.6:0/2589765919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f297415d730 msgr2=0x7f29740ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f297415d730 0x7f29740ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 10 monclient: _finish_hunting 0 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 1 monclient: found mon.noname-a 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.303+0000 7f297a015640 1 -- 10.20.193.6:0/2589765919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29741f3910 con 0x7f29741527f0 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.304+0000 7f297a015640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f29741f32b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2964009a70 tx=0x7f29640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.304+0000 7f295affd640 10 client.?.objecter ms_handle_connect 0x7f29741527f0 2026-02-22T23:21:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.304+0000 7f295affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 1 -- 10.20.193.6:0/2589765919 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964047070 con 0x7f29741527f0 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: dump: 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: _finish_auth 0 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 5 monclient: authenticate success, global_id 4194 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 monclient: _renew_subs 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741f3bf0 con 0x7f29741527f0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 client.4194.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 monclient: _renew_subs 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29740c9c80 con 0x7f29741527f0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 1 librados: init done 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974152bd0 con 0x7f29741527f0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 1 -- 10.20.193.6:0/2589765919 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296402fb20 con 0x7f29741527f0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 1 -- 10.20.193.6:0/2589765919 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296403c040 con 0x7f29741527f0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f2978812640 4 set_mon_vals no callback set 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.305+0000 7f295affd640 10 monclient: dump: 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:45.313 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:21:45.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.409+0000 7f295affd640 1 -- 10.20.193.6:0/2589765919 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f2964042830 con 0x7f29741527f0 2026-02-22T23:21:45.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.409+0000 7f295affd640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2944078d70 0x7f294407b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:45.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.410+0000 7f2979814640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2944078d70 0x7f294407b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:45.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.410+0000 7f2979814640 10 monclient: get_auth_request con 0x7f2944078d70 auth_method 0 2026-02-22T23:21:45.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.410+0000 7f295affd640 10 client.4194.objecter ms_handle_connect 0x7f2944078d70 2026-02-22T23:21:45.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.410+0000 7f2979814640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2944078d70 0x7f294407b230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f297415d250 tx=0x7f2968007f30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:45.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 1 -- 10.20.193.6:0/2589765919 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f29640829f0 con 0x7f29741527f0 2026-02-22T23:21:45.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 10 client.4194.objecter ms_dispatch2 0x7f2974000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:21:45.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 3 client.4194.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:21:45.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 3 client.4194.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:21:45.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 20 client.4194.objecter dump_active .. 0 homeless 2026-02-22T23:21:45.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 1 -- 10.20.193.6:0/2589765919 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2964052080 con 0x7f29741527f0 2026-02-22T23:21:45.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:45.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.417+0000 7f295affd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:45.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.504+0000 7f297c2a0640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:45.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.504+0000 7f297c2a0640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:45.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.504+0000 7f297c2a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:45.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.504+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29740cee90 con 0x7f29741527f0 2026-02-22T23:21:45.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.529+0000 7f295affd640 1 -- 10.20.193.6:0/2589765919 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f29640869b0 con 0x7f29741527f0 2026-02-22T23:21:45.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.529+0000 7f295affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:45.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.529+0000 7f295affd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:45.531 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:21:45.531 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:21:45.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 10 librados: watch_flush enter 2026-02-22T23:21:45.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 10 librados: watch_flush exit 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 20 client.4194.objecter shutdown clearing up homeless session... 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 10 client.4194.objecter successfully canceled tick 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2944078d70 msgr2=0x7f294407b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2944078d70 0x7f294407b230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f297415d250 tx=0x7f2968007f30 comp rx=0 tx=0).stop 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 10 monclient: shutdown 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 msgr2=0x7f29741f32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f29741f32b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2964009a70 tx=0x7f29640047c0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 shutdown_connections 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2944078d70 0x7f294407b230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f297415d730 0x7f29740ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f297415d2c0 0x7f29741f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 --2- 10.20.193.6:0/2589765919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f29741527f0 0x7f29741f32b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:45.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 >> 10.20.193.6:0/2589765919 conn(0x7f29740ce730 msgr2=0x7f29740d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:45.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 shutdown_connections 2026-02-22T23:21:45.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 -- 10.20.193.6:0/2589765919 wait complete. 2026-02-22T23:21:45.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:21:45.531+0000 7f297c2a0640 1 librados: shutdown 2026-02-22T23:21:45.542 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-22T23:21:45.542 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-22T23:21:47.155 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:21:47.218 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.217+0000 7f3a38ebd640 10 monclient: get_monmap_and_config 2026-02-22T23:21:47.218 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.217+0000 7f3a38ebd640 10 monclient: build_initial_monmap 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.217+0000 7f3a38ebd640 10 monclient: monmap: 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:21:47.218243+0000 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:21:47.218243+0000 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:47.219 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 1 Processor -- start 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 1 -- start start 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient: init 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient: _add_conns ranks=[2,5,1,3,4,0] 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3a341539a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f3a34153d80 0x7f3a34074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient(hunting): picked mon.noname-c con 0x7f3a34153d80 addr v2:10.20.193.80:3300/0 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3a340c8b00 addr v1:10.20.193.40:6789/0 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a340c5710 con 0x7f3a340c8b00 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a340c59f0 con 0x7f3a341539a0 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3a340c5cd0 con 0x7f3a34153d80 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.218+0000 7f3a38ebd640 10 monclient(hunting): authenticate will time out at 981.945602s 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.219+0000 7f3a32d76640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f3a340c8b00 0x7f3a340c8f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:52998/0 (socket says 10.20.193.6:52998) 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.219+0000 7f3a32d76640 1 -- 10.20.193.6:0/736287439 learned_addr learned my addr 10.20.193.6:0/736287439 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:21:47.220 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.219+0000 7f3a31d74640 1 --2- 10.20.193.6:0/736287439 >> v2:10.20.193.80:3300/0 conn(0x7f3a34153d80 0x7f3a34074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.219+0000 7f3a31d74640 10 monclient(hunting): get_auth_request con 0x7f3a34153d80 auth_method 0 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.219+0000 7f3a31d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.219+0000 7f3a31d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.219+0000 7f3a31d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:47.221 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31573640 1 -- 10.20.193.6:0/736287439 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057748481 0 0) 0x7f3a340c5710 con 0x7f3a340c8b00 2026-02-22T23:21:47.222 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31573640 10 monclient(hunting): _negotiate 2026-02-22T23:21:47.222 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:47.222 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:47.222 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31573640 10 monclient(hunting): my global_id is 4201 2026-02-22T23:21:47.222 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.220+0000 7f3a31573640 1 -- 10.20.193.6:0/736287439 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a20003620 con 0x7f3a340c8b00 2026-02-22T23:21:47.222 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 10 monclient(hunting): handle_auth_done global_id 4226 payload 274 2026-02-22T23:21:47.222 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 1 -- 10.20.193.6:0/736287439 >> v1:10.20.193.80:6789/0 conn(0x7f3a341539a0 legacy=0x7f3a340c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:47.223 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 1 -- 10.20.193.6:0/736287439 >> v1:10.20.193.40:6789/0 conn(0x7f3a340c8b00 legacy=0x7f3a340c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:47.223 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 10 monclient: _finish_hunting 0 2026-02-22T23:21:47.223 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 1 monclient: found mon.noname-c 2026-02-22T23:21:47.223 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:47.223 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:47.223 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.221+0000 7f3a31d74640 1 -- 10.20.193.6:0/736287439 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a340c8020 con 0x7f3a34153d80 2026-02-22T23:21:47.225 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.224+0000 7f3a31d74640 1 --2- 10.20.193.6:0/736287439 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34074960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0098a0 tx=0x7f3a1c02edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58067f60c54aca8 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 1 -- 10.20.193.6:0/736287439 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c03c070 con 0x7f3a34153d80 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: dump: 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: _finish_auth 0 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: _check_auth_tickets 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 1 -- 10.20.193.6:0/736287439 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1c02fae0 con 0x7f3a34153d80 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 1 -- 10.20.193.6:0/736287439 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c037440 con 0x7f3a34153d80 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a38ebd640 5 monclient: authenticate success, global_id 4226 2026-02-22T23:21:47.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a38ebd640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a38ebd640 4 set_mon_vals no callback set 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a31573640 10 monclient: dump: 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a38ebd640 10 monclient: shutdown 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a38ebd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.226+0000 7f3a38ebd640 1 -- 10.20.193.6:0/736287439 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 msgr2=0x7f3a34074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/736287439 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34074960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0098a0 tx=0x7f3a1c02edd0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 -- 10.20.193.6:0/736287439 shutdown_connections 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/736287439 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34074960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 -- 10.20.193.6:0/736287439 >> 10.20.193.6:0/736287439 conn(0x7f3a340c9f40 msgr2=0x7f3a340ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 -- 10.20.193.6:0/736287439 shutdown_connections 2026-02-22T23:21:47.229 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 -- 10.20.193.6:0/736287439 wait complete. 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 10 monclient: build_initial_monmap 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 10 monclient: monmap: 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 librados: starting msgr at 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 librados: starting objecter 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 Processor -- start 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 -- start start 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 librados: setting wanted keys 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 1 librados: calling monclient init 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.227+0000 7f3a38ebd640 10 monclient: init 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3a340c8b00 0x7f3a341f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): picked mon.noname-a con 0x7f3a340c8b00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3a341539a0 0x7f3a3406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): picked mon.noname-b con 0x7f3a341539a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): picked mon.noname-c con 0x7f3a34153d80 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:47.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340c52d0 con 0x7f3a340c8b00 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340c85e0 con 0x7f3a341539a0 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3a340c5450 con 0x7f3a34153d80 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a38ebd640 10 monclient(hunting): authenticate will time out at 981.955154s 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a31d74640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3a341539a0 0x7f3a3406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32d76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3a340c8b00 0x7f3a341f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32d76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:54436/0 (socket says 10.20.193.6:54436) 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32d76640 1 -- 10.20.193.6:0/1660655492 learned_addr learned my addr 10.20.193.6:0/1660655492 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32d76640 10 monclient(hunting): get_auth_request con 0x7f3a34153d80 auth_method 0 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a31d74640 10 monclient(hunting): get_auth_request con 0x7f3a341539a0 auth_method 0 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a31d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a31d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a31d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32575640 10 monclient(hunting): get_auth_request con 0x7f3a340c8b00 auth_method 0 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.228+0000 7f3a32575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a32d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a32d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a32d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a31d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a31d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a31d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a32575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:47.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a32575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.229+0000 7f3a32575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 10 monclient(hunting): handle_auth_done global_id 4232 payload 824 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 1 -- 10.20.193.6:0/1660655492 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3a341539a0 msgr2=0x7f3a3406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3a341539a0 0x7f3a3406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 1 -- 10.20.193.6:0/1660655492 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3a340c8b00 msgr2=0x7f3a341f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3a340c8b00 0x7f3a341f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 10 monclient: _finish_hunting 0 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 1 monclient: found mon.noname-c 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.230+0000 7f3a32d76640 1 -- 10.20.193.6:0/1660655492 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a341f7830 con 0x7f3a34153d80 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.231+0000 7f3a32d76640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34073120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400cb70 tx=0x7f3a2400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.231+0000 7f3a1affd640 10 client.?.objecter ms_handle_connect 0x7f3a34153d80 2026-02-22T23:21:47.233 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.231+0000 7f3a1affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 1 -- 10.20.193.6:0/1660655492 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400ad50 con 0x7f3a34153d80 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: dump: 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: _finish_auth 0 2026-02-22T23:21:47.234 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: _check_auth_tickets 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 5 monclient: authenticate success, global_id 4232 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 monclient: _renew_subs 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a34155fa0 con 0x7f3a34153d80 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 client.4232.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 monclient: _renew_subs 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a34156560 con 0x7f3a34153d80 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 1 librados: init done 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 1 -- 10.20.193.6:0/1660655492 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2401b400 con 0x7f3a34153d80 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 1 -- 10.20.193.6:0/1660655492 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2401b7f0 con 0x7f3a34153d80 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a30d72640 4 set_mon_vals no callback set 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a1affd640 10 monclient: dump: 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:47.235 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:47.236 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:47.236 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:47.236 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:47.236 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.233+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a340c8f50 con 0x7f3a34153d80 2026-02-22T23:21:47.246 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.245+0000 7f3a1affd640 1 -- 10.20.193.6:0/1660655492 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f3a2401b990 con 0x7f3a34153d80 2026-02-22T23:21:47.246 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.245+0000 7f3a1affd640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3a00078d70 0x7f3a0007b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:47.246 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.245+0000 7f3a32575640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3a00078d70 0x7f3a0007b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:47.247 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.245+0000 7f3a32575640 10 monclient: get_auth_request con 0x7f3a00078d70 auth_method 0 2026-02-22T23:21:47.247 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.246+0000 7f3a32575640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3a00078d70 0x7f3a0007b230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a28002410 tx=0x7f3a28007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:47.247 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.246+0000 7f3a1affd640 10 client.4232.objecter ms_handle_connect 0x7f3a00078d70 2026-02-22T23:21:47.248 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.247+0000 7f3a1affd640 1 -- 10.20.193.6:0/1660655492 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f3a2405e940 con 0x7f3a34153d80 2026-02-22T23:21:47.248 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.247+0000 7f3a1affd640 10 client.4232.objecter ms_dispatch2 0x7f3a34000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:21:47.248 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.247+0000 7f3a1affd640 3 client.4232.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:21:47.248 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.247+0000 7f3a1affd640 3 client.4232.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:21:47.248 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.247+0000 7f3a1affd640 20 client.4232.objecter dump_active .. 0 homeless 2026-02-22T23:21:47.350 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.349+0000 7f3a1affd640 1 -- 10.20.193.6:0/1660655492 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3a24062900 con 0x7f3a34153d80 2026-02-22T23:21:47.350 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.349+0000 7f3a1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:47.350 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.349+0000 7f3a1affd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:47.438 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.436+0000 7f3a38ebd640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:47.438 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.436+0000 7f3a38ebd640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:47.438 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.436+0000 7f3a38ebd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:21:47.438 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.436+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a340cc260 con 0x7f3a34153d80 2026-02-22T23:21:47.466 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.464+0000 7f3a1affd640 1 -- 10.20.193.6:0/1660655492 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f3a24066c00 con 0x7f3a34153d80 2026-02-22T23:21:47.466 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.464+0000 7f3a1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:47.466 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.464+0000 7f3a1affd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:47.466 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:21:47.466 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:21:47.469 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 10 librados: watch_flush enter 2026-02-22T23:21:47.469 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 10 librados: watch_flush exit 2026-02-22T23:21:47.469 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 20 client.4232.objecter shutdown clearing up homeless session... 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 10 client.4232.objecter successfully canceled tick 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3a00078d70 msgr2=0x7f3a0007b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3a00078d70 0x7f3a0007b230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a28002410 tx=0x7f3a28007ef0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 10 monclient: shutdown 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 msgr2=0x7f3a34073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34073120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400cb70 tx=0x7f3a2400a430 comp rx=0 tx=0).stop 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 shutdown_connections 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3a00078d70 0x7f3a0007b230 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3a34153d80 0x7f3a34073120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.470 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3a341539a0 0x7f3a3406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.471 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 --2- 10.20.193.6:0/1660655492 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3a340c8b00 0x7f3a341f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:47.471 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 >> 10.20.193.6:0/1660655492 conn(0x7f3a340c9f40 msgr2=0x7f3a3406b730 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:47.471 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 shutdown_connections 2026-02-22T23:21:47.471 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 -- 10.20.193.6:0/1660655492 wait complete. 2026-02-22T23:21:47.471 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:47.467+0000 7f3a38ebd640 1 librados: shutdown 2026-02-22T23:21:47.476 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:21:54.931 INFO:tasks.ceph.osd.4.trial006.stderr:--00:00:00:09.754 45051-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:54.931 INFO:tasks.ceph.osd.8.trial006.stderr:--00:00:00:09.754 45052-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:54.934 INFO:tasks.ceph.osd.0.trial006.stderr:--00:00:00:09.760 45050-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:54.950 INFO:tasks.ceph.osd.5.trial040.stderr:--00:00:00:09.762 44610-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:54.951 INFO:tasks.ceph.osd.1.trial040.stderr:--00:00:00:09.766 44609-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:54.952 INFO:tasks.ceph.osd.9.trial040.stderr:--00:00:00:09.759 44611-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:54.989 INFO:tasks.ceph.osd.12.trial006.stderr:--00:00:00:09.808 45053-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.023 INFO:tasks.ceph.osd.3.trial189.stderr:--00:00:00:09.750 44905-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.044 INFO:tasks.ceph.osd.15.trial189.stderr:--00:00:00:09.771 44908-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.045 INFO:tasks.ceph.osd.7.trial189.stderr:--00:00:00:09.772 44906-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.045 INFO:tasks.ceph.osd.11.trial189.stderr:--00:00:00:09.772 44907-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.063 INFO:tasks.ceph.osd.13.trial040.stderr:--00:00:00:09.815 44638-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.070 INFO:tasks.ceph.osd.6.trial080.stderr:--00:00:00:09.810 44713-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.072 INFO:tasks.ceph.osd.10.trial080.stderr:--00:00:00:09.812 44712-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.072 INFO:tasks.ceph.osd.2.trial080.stderr:--00:00:00:09.814 44711-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.074 INFO:tasks.ceph.osd.14.trial080.stderr:--00:00:00:09.811 44714-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-22T23:21:55.086 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:21:55.152 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.149+0000 7fa10c103640 10 monclient: get_monmap_and_config 2026-02-22T23:21:55.152 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.149+0000 7fa10c103640 10 monclient: build_initial_monmap 2026-02-22T23:21:55.152 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.149+0000 7fa10c103640 10 monclient: monmap: 2026-02-22T23:21:55.152 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:21:55.152 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:21:55.151485+0000 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:21:55.151485+0000 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:21:55.153 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:55.156 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.150+0000 7fa10c103640 1 Processor -- start 2026-02-22T23:21:55.156 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.150+0000 7fa10c103640 1 -- start start 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.150+0000 7fa10c103640 10 monclient: init 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient: _add_conns ranks=[3,0,2,5,4,1] 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa1041539a0 0x7fa1040c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient(hunting): picked mon.noname-a con 0x7fa1041539a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa104153d80 addr v1:10.20.193.6:6789/0 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa1040c8b00 addr v1:10.20.193.80:6789/0 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa104075070 con 0x7fa104153d80 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1040c5710 con 0x7fa1040c8b00 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa1040c5cd0 con 0x7fa1041539a0 2026-02-22T23:21:55.157 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10c103640 10 monclient(hunting): authenticate will time out at 989.879267s 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa109e78640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa1041539a0 0x7fa1040c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10a679640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fa1040c8b00 0x7fa1040c8f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:37752/0 (socket says 10.20.193.6:37752) 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.151+0000 7fa10a679640 1 -- 10.20.193.6:0/2328976745 learned_addr learned my addr 10.20.193.6:0/2328976745 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.152+0000 7fa109e78640 10 monclient(hunting): get_auth_request con 0x7fa1041539a0 auth_method 0 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.152+0000 7fa109e78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.152+0000 7fa109e78640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.152+0000 7fa109e78640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.152+0000 7fa109e78640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.152+0000 7fa109e78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.152+0000 7fa109e78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:55.158 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.153+0000 7fa108e76640 1 -- 10.20.193.6:0/2328976745 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1335776109 0 0) 0x7fa1040c5710 con 0x7fa1040c8b00 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.153+0000 7fa108e76640 10 monclient(hunting): _negotiate 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.153+0000 7fa108e76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.153+0000 7fa108e76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.153+0000 7fa108e76640 10 monclient(hunting): my global_id is 4235 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.153+0000 7fa108e76640 1 -- 10.20.193.6:0/2328976745 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0ec003620 con 0x7fa1040c8b00 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 10 monclient(hunting): handle_auth_done global_id 4200 payload 274 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 1 -- 10.20.193.6:0/2328976745 >> v1:10.20.193.80:6789/0 conn(0x7fa1040c8b00 legacy=0x7fa1040c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 1 -- 10.20.193.6:0/2328976745 >> v1:10.20.193.6:6789/0 conn(0x7fa104153d80 legacy=0x7fa104074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 10 monclient: _finish_hunting 0 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 1 monclient: found mon.noname-a 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:55.159 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:55.160 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.154+0000 7fa109e78640 1 -- 10.20.193.6:0/2328976745 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1040c59f0 con 0x7fa1041539a0 2026-02-22T23:21:55.160 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.156+0000 7fa109e78640 1 --2- 10.20.193.6:0/2328976745 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1041539a0 0x7fa1040c7270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40098a0 tx=0x7fa0f402edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c6ab510d18aec5 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:55.165 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 1 -- 10.20.193.6:0/2328976745 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f403c070 con 0x7fa1041539a0 2026-02-22T23:21:55.165 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:55.165 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:55.165 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: dump: 2026-02-22T23:21:55.165 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:55.165 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:55.165 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: _finish_auth 0 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: _check_auth_tickets 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 1 -- 10.20.193.6:0/2328976745 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f402fae0 con 0x7fa1041539a0 2026-02-22T23:21:55.166 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 5 monclient: authenticate success, global_id 4200 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 1 -- 10.20.193.6:0/2328976745 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4037440 con 0x7fa1041539a0 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 10 monclient: get_monmap_and_config success 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 4 set_mon_vals no callback set 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa108e76640 10 monclient: dump: 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.167 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 10 monclient: shutdown 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 1 -- 10.20.193.6:0/2328976745 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1041539a0 msgr2=0x7fa1040c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 1 --2- 10.20.193.6:0/2328976745 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1041539a0 0x7fa1040c7270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40098a0 tx=0x7fa0f402edd0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 1 -- 10.20.193.6:0/2328976745 shutdown_connections 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 1 --2- 10.20.193.6:0/2328976745 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1041539a0 0x7fa1040c7270 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.168 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 1 -- 10.20.193.6:0/2328976745 >> 10.20.193.6:0/2328976745 conn(0x7fa1040c9f40 msgr2=0x7fa1040ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 1 -- 10.20.193.6:0/2328976745 shutdown_connections 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.159+0000 7fa10c103640 1 -- 10.20.193.6:0/2328976745 wait complete. 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient: build_initial_monmap 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient: monmap: 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:21:55.169 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 librados: starting msgr at 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 librados: starting objecter 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 Processor -- start 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 -- start start 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 librados: setting wanted keys 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 librados: calling monclient init 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient: init 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient: _reopen_session rank -1 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1040c8b00 0x7fa1040726d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:55.170 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): picked mon.noname-c con 0x7fa1040c8b00 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1041539a0 0x7fa1041f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): picked mon.noname-b con 0x7fa1041539a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa104153d80 0x7fa1041f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): picked mon.noname-a con 0x7fa104153d80 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040c85e0 con 0x7fa104153d80 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040c8300 con 0x7fa1041539a0 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1040c77b0 con 0x7fa1040c8b00 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): _renew_subs 2026-02-22T23:21:55.171 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.160+0000 7fa10c103640 10 monclient(hunting): authenticate will time out at 989.888875s 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa104153d80 0x7fa1041f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1040c8b00 0x7fa1040726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa104153d80 0x7fa1041f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:50478/0 (socket says 10.20.193.6:50478) 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 1 -- 10.20.193.6:0/455245557 learned_addr learned my addr 10.20.193.6:0/455245557 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 10 monclient(hunting): get_auth_request con 0x7fa104153d80 auth_method 0 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 10 monclient(hunting): get_auth_request con 0x7fa1040c8b00 auth_method 0 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:55.172 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109677640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1041539a0 0x7fa1041f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa10a679640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109677640 10 monclient(hunting): get_auth_request con 0x7fa1041539a0 auth_method 0 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109677640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109677640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109677640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.161+0000 7fa109e78640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.162+0000 7fa109677640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:21:55.173 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.162+0000 7fa109677640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:21:55.174 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.162+0000 7fa109677640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:21:55.176 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 10 monclient(hunting): handle_auth_done global_id 4206 payload 824 2026-02-22T23:21:55.176 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 1 -- 10.20.193.6:0/455245557 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1040c8b00 msgr2=0x7fa1040726d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:55.176 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1040c8b00 0x7fa1040726d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.176 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 1 -- 10.20.193.6:0/455245557 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1041539a0 msgr2=0x7fa1041f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:55.176 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1041539a0 0x7fa1041f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.176 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 10 monclient: _finish_hunting 0 2026-02-22T23:21:55.176 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 1 monclient: found mon.noname-a 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.163+0000 7fa10a679640 1 -- 10.20.193.6:0/455245557 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa104072d30 con 0x7fa104153d80 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.164+0000 7fa10a679640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa104153d80 0x7fa1041f6f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa10000cb40 tx=0x7fa10000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.164+0000 7fa0f27fc640 10 client.?.objecter ms_handle_connect 0x7fa104153d80 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.164+0000 7fa0f27fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 1 -- 10.20.193.6:0/455245557 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000ad50 con 0x7fa104153d80 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: dump: 2026-02-22T23:21:55.177 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: _finish_auth 0 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: _check_auth_tickets 2026-02-22T23:21:55.178 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 1 -- 10.20.193.6:0/455245557 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa100012400 con 0x7fa104153d80 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 1 -- 10.20.193.6:0/455245557 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000127f0 con 0x7fa104153d80 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f27fc640 10 monclient: dump: 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:21:55.179 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa0f3fff640 4 set_mon_vals no callback set 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 5 monclient: authenticate success, global_id 4206 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 10 monclient: _renew_subs 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa104073010 con 0x7fa104153d80 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 10 client.4206.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 10 monclient: _renew_subs 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1040735d0 con 0x7fa104153d80 2026-02-22T23:21:55.180 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.166+0000 7fa10c103640 1 librados: init done 2026-02-22T23:21:55.181 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.167+0000 7fa10c103640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:55.181 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.167+0000 7fa10c103640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:55.181 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.167+0000 7fa10c103640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:55.181 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.167+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1040c52d0 con 0x7fa104153d80 2026-02-22T23:21:55.181 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.179+0000 7fa0f27fc640 1 -- 10.20.193.6:0/455245557 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7fa100012990 con 0x7fa104153d80 2026-02-22T23:21:55.182 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.180+0000 7fa0f27fc640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0d4078d70 0x7fa0d407b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:21:55.182 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.180+0000 7fa109e78640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0d4078d70 0x7fa0d407b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:21:55.182 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.180+0000 7fa109e78640 10 monclient: get_auth_request con 0x7fa0d4078d70 auth_method 0 2026-02-22T23:21:55.183 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.181+0000 7fa0f27fc640 10 client.4206.objecter ms_handle_connect 0x7fa0d4078d70 2026-02-22T23:21:55.183 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.180+0000 7fa109e78640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0d4078d70 0x7fa0d407b230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4039000 tx=0x7fa0f4033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:21:55.184 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.181+0000 7fa0f27fc640 1 -- 10.20.193.6:0/455245557 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7fa10005e860 con 0x7fa104153d80 2026-02-22T23:21:55.184 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.181+0000 7fa0f27fc640 10 client.4206.objecter ms_dispatch2 0x7fa104000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:21:55.184 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.181+0000 7fa0f27fc640 3 client.4206.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:21:55.184 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.181+0000 7fa0f27fc640 3 client.4206.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:21:55.184 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.182+0000 7fa0f27fc640 20 client.4206.objecter dump_active .. 0 homeless 2026-02-22T23:21:55.284 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.282+0000 7fa0f27fc640 1 -- 10.20.193.6:0/455245557 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa100062820 con 0x7fa104153d80 2026-02-22T23:21:55.284 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.282+0000 7fa0f27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:21:55.285 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.282+0000 7fa0f27fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:21:55.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.367+0000 7fa10c103640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:55.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.367+0000 7fa10c103640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:55.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.367+0000 7fa10c103640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:21:55.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.367+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa10415aa40 con 0x7fa104153d80 2026-02-22T23:21:55.380 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.378+0000 7fa0f27fc640 1 -- 10.20.193.6:0/455245557 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7fa100066b20 con 0x7fa104153d80 2026-02-22T23:21:55.380 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.378+0000 7fa0f27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:21:55.380 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.378+0000 7fa0f27fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:21:55.380 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:21:55.381 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:21:55.382 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 10 librados: watch_flush enter 2026-02-22T23:21:55.382 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 10 librados: watch_flush exit 2026-02-22T23:21:55.382 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 20 client.4206.objecter shutdown clearing up homeless session... 2026-02-22T23:21:55.382 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 10 client.4206.objecter successfully canceled tick 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0d4078d70 msgr2=0x7fa0d407b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0d4078d70 0x7fa0d407b230 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4039000 tx=0x7fa0f4033000 comp rx=0 tx=0).stop 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 10 monclient: shutdown 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa104153d80 msgr2=0x7fa1041f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa104153d80 0x7fa1041f6f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa10000cb40 tx=0x7fa10000a430 comp rx=0 tx=0).stop 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 shutdown_connections 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0d4078d70 0x7fa0d407b230 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa104153d80 0x7fa1041f6f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1041539a0 0x7fa1041f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 --2- 10.20.193.6:0/455245557 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1040c8b00 0x7fa1040726d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:21:55.383 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 >> 10.20.193.6:0/455245557 conn(0x7fa1040c9f40 msgr2=0x7fa10414dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:21:55.384 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 shutdown_connections 2026-02-22T23:21:55.384 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 -- 10.20.193.6:0/455245557 wait complete. 2026-02-22T23:21:55.384 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:21:55.379+0000 7fa10c103640 1 librados: shutdown 2026-02-22T23:21:55.389 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:21:59.508 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:14.330 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.510 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:14.334 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.511 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:14.337 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.545 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:14.357 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.549 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:14.291 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.568 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:14.295 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.574 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:14.381 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.575 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:14.395 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.587 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:14.403 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.616 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:14.353 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.617 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:14.357 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.625 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:14.365 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.642 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:14.369 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.642 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:14.370 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.655 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:14.407 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.667 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:14.394 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.793 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:14.616 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.799 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:14.626 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.810 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:14.633 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.820 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:14.642 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.829 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:14.641 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.832 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:14.574 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.835 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:14.659 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.849 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:14.661 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.851 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:14.675 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.854 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:14.581 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.856 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:14.598 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.863 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:14.682 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.869 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:14.676 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.874 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:14.686 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.881 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:14.697 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.884 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:14.704 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.890 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:14.697 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.899 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:14.636 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.906 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:14.726 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.912 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:14.652 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.914 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:14.721 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.915 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:14.655 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.921 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:14.658 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.928 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:14.656 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.932 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:14.659 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.937 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:14.677 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.937 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:14.677 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.948 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:14.699 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.953 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:14.680 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.960 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:14.687 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.961 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:14.701 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.961 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:14.701 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:21:59.973 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:14.724 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.063 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:14.889 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.088 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:14.910 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.118 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:14.941 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.119 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:14.861 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.122 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:14.849 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.137 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:14.949 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.158 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:14.974 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.175 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:14.994 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.183 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:14.999 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.185 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:14.922 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.188 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:14.995 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.198 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:14.925 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.226 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:14.953 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.227 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:14.967 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.231 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:14.971 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.234 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:14.962 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.236 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:14.988 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.258 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:14.986 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.328 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:15.154 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.358 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:15.181 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.383 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:15.125 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.386 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:15.113 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.389 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:15.212 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.404 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:15.228 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.409 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:15.221 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.424 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:15.236 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.447 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:15.266 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.453 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:15.190 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.457 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:15.273 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.462 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:15.281 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.462 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:15.189 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.465 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:15.272 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.480 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:15.287 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.493 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:15.221 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.500 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:15.240 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.507 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:15.259 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.510 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:15.250 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.516 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:15.256 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.522 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:15.250 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.528 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:15.268 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.530 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:15.282 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.545 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:15.297 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.592 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:15.418 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.630 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:15.454 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.646 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:15.470 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.648 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:15.390 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.650 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:15.377 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.663 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:15.405 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.718 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:15.455 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.726 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:15.542 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.727 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:15.453 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.733 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:15.470 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.741 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:15.557 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.758 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:15.486 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.774 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:15.501 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.802 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:15.529 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.816 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:15.544 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.857 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:15.684 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.873 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:15.699 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.916 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:15.643 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.931 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:15.659 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.939 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:22:00.934+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:00.953 INFO:tasks.ceph.osd.5.trial040.stderr:2026-02-22T23:22:00.945+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:00.992 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:15.719 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:00.995 INFO:tasks.ceph.osd.12.trial006.stderr:2026-02-22T23:22:00.990+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.007 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:15.734 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:01.012 INFO:tasks.ceph.osd.9.trial040.stderr:2026-02-22T23:22:01.004+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.047 INFO:tasks.ceph.osd.6.trial080.stderr:2026-02-22T23:22:01.043+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.066 INFO:tasks.ceph.osd.10.trial080.stderr:2026-02-22T23:22:01.062+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.076 INFO:tasks.ceph.osd.13.trial040.stderr:2026-02-22T23:22:01.068+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.184 INFO:tasks.ceph.osd.8.trial006.stderr:2026-02-22T23:22:01.179+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.197 INFO:tasks.ceph.osd.2.trial080.stderr:2026-02-22T23:22:01.193+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.271 INFO:tasks.ceph.osd.1.trial040.stderr:2026-02-22T23:22:01.264+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.272 INFO:tasks.ceph.osd.14.trial080.stderr:2026-02-22T23:22:01.267+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.301 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-22T23:22:01.298+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.369 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-22T23:22:01.367+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.414 INFO:tasks.ceph.osd.0.trial006.stderr:2026-02-22T23:22:01.408+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.477 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-22T23:22:01.474+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:01.550 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-22T23:22:01.547+0000 77b8940 -1 Falling back to public interface 2026-02-22T23:22:03.000 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:03.064 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.061+0000 7f4a698bf640 10 monclient: get_monmap_and_config 2026-02-22T23:22:03.064 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.061+0000 7f4a698bf640 10 monclient: build_initial_monmap 2026-02-22T23:22:03.064 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.061+0000 7f4a698bf640 10 monclient: monmap: 2026-02-22T23:22:03.064 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:03.064 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:03.063318+0000 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:03.063318+0000 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:03.065 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 1 Processor -- start 2026-02-22T23:22:03.066 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 1 -- start start 2026-02-22T23:22:03.066 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient: init 2026-02-22T23:22:03.068 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient: _add_conns ranks=[4,0,5,1,2,3] 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4a64152a40 0x7f4a640c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient(hunting): picked mon.noname-b con 0x7f4a64152a40 addr v2:10.20.193.40:3300/0 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4a64152e20 addr v1:10.20.193.6:6789/0 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4a6406dd40 0x7f4a6406e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient(hunting): picked mon.noname-c con 0x7f4a6406dd40 addr v2:10.20.193.80:3300/0 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a6406e6d0 con 0x7f4a64152e20 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640c8550 con 0x7f4a64152a40 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:03.069 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4a6406d5d0 con 0x7f4a6406dd40 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.062+0000 7f4a698bf640 10 monclient(hunting): authenticate will time out at 997.790675s 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a627fc640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f4a64152e20 0x7f4a640c7e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:59622/0 (socket says 10.20.193.6:59622) 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a627fc640 1 -- 10.20.193.6:0/2383926799 learned_addr learned my addr 10.20.193.6:0/2383926799 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4a64152a40 0x7f4a640c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a637fe640 1 --2- 10.20.193.6:0/2383926799 >> v2:10.20.193.80:3300/0 conn(0x7f4a6406dd40 0x7f4a6406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 10 monclient(hunting): get_auth_request con 0x7f4a64152a40 auth_method 0 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a637fe640 10 monclient(hunting): get_auth_request con 0x7f4a6406dd40 auth_method 0 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a637fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:03.070 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.063+0000 7f4a62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 10 monclient(hunting): handle_auth_done global_id 4258 payload 274 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 1 -- 10.20.193.6:0/2383926799 >> v2:10.20.193.80:3300/0 conn(0x7f4a6406dd40 msgr2=0x7f4a6406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 1 --2- 10.20.193.6:0/2383926799 >> v2:10.20.193.80:3300/0 conn(0x7f4a6406dd40 0x7f4a6406e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 1 -- 10.20.193.6:0/2383926799 >> v1:10.20.193.6:6789/0 conn(0x7f4a64152e20 legacy=0x7f4a640c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 1 monclient: found mon.noname-b 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:03.071 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:22:03.072 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.064+0000 7f4a62ffd640 1 -- 10.20.193.6:0/2383926799 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640c6d70 con 0x7f4a64152a40 2026-02-22T23:22:03.072 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.066+0000 7f4a62ffd640 1 --2- 10.20.193.6:0/2383926799 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152a40 0x7f4a640c99a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009d90 tx=0x7f4a5802f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=371606c331d4bad1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 1 -- 10.20.193.6:0/2383926799 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5803c070 con 0x7f4a64152a40 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: dump: 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:03.076 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: _finish_auth 0 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 1 -- 10.20.193.6:0/2383926799 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a58037440 con 0x7f4a64152a40 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 5 monclient: authenticate success, global_id 4258 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 1 -- 10.20.193.6:0/2383926799 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58037810 con 0x7f4a64152a40 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 4 set_mon_vals no callback set 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:22:03.077 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a61ffb640 10 monclient: dump: 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 10 monclient: shutdown 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 -- 10.20.193.6:0/2383926799 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152a40 msgr2=0x7f4a640c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 --2- 10.20.193.6:0/2383926799 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152a40 0x7f4a640c99a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009d90 tx=0x7f4a5802f1e0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 -- 10.20.193.6:0/2383926799 shutdown_connections 2026-02-22T23:22:03.078 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 --2- 10.20.193.6:0/2383926799 >> v2:10.20.193.80:3300/0 conn(0x7f4a6406dd40 0x7f4a6406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 --2- 10.20.193.6:0/2383926799 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152a40 0x7f4a640c99a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 -- 10.20.193.6:0/2383926799 >> 10.20.193.6:0/2383926799 conn(0x7f4a640c9f40 msgr2=0x7f4a640ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 -- 10.20.193.6:0/2383926799 shutdown_connections 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.068+0000 7f4a698bf640 1 -- 10.20.193.6:0/2383926799 wait complete. 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient: build_initial_monmap 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient: monmap: 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:03.079 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 librados: starting msgr at 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 librados: starting objecter 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 Processor -- start 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 -- start start 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 librados: setting wanted keys 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 librados: calling monclient init 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient: init 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4a6406dd40 0x7f4a641f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): picked mon.noname-a con 0x7f4a6406dd40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4a64152a40 0x7f4a641f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): picked mon.noname-c con 0x7f4a64152a40 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152e20 0x7f4a6415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): picked mon.noname-b con 0x7f4a64152e20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640c7610 con 0x7f4a6406dd40 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:03.080 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640c7330 con 0x7f4a64152e20 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6406d820 con 0x7f4a64152a40 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.069+0000 7f4a698bf640 10 monclient(hunting): authenticate will time out at 997.797864s 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4a6406dd40 0x7f4a641f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4a64152a40 0x7f4a641f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a637fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152e20 0x7f4a6415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4a6406dd40 0x7f4a641f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:35344/0 (socket says 10.20.193.6:35344) 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 1 -- 10.20.193.6:0/1007802713 learned_addr learned my addr 10.20.193.6:0/1007802713 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 10 monclient(hunting): get_auth_request con 0x7f4a6406dd40 auth_method 0 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 10 monclient(hunting): get_auth_request con 0x7f4a64152a40 auth_method 0 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:03.081 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a637fe640 10 monclient(hunting): get_auth_request con 0x7f4a64152e20 auth_method 0 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a637fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a62ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.070+0000 7f4a637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.071+0000 7f4a637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 10 monclient(hunting): handle_auth_done global_id 4261 payload 824 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 1 -- 10.20.193.6:0/1007802713 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4a64152a40 msgr2=0x7f4a641f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4a64152a40 0x7f4a641f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.082 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 1 -- 10.20.193.6:0/1007802713 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4a6406dd40 msgr2=0x7f4a641f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4a6406dd40 0x7f4a641f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 10 monclient: _finish_hunting 0 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 1 monclient: found mon.noname-b 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.072+0000 7f4a637fe640 1 -- 10.20.193.6:0/1007802713 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a641f5f50 con 0x7f4a64152e20 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.073+0000 7f4a637fe640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152e20 0x7f4a6415ab80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e960 tx=0x7f4a5400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.073+0000 7f4a3bfff640 10 client.?.objecter ms_handle_connect 0x7f4a64152e20 2026-02-22T23:22:03.083 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.073+0000 7f4a3bfff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:03.084 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 1 -- 10.20.193.6:0/1007802713 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54017070 con 0x7f4a64152e20 2026-02-22T23:22:03.084 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:03.084 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:22:03.084 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: dump: 2026-02-22T23:22:03.084 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: _finish_auth 0 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 1 -- 10.20.193.6:0/1007802713 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5400c980 con 0x7f4a64152e20 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 5 monclient: authenticate success, global_id 4261 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 10 monclient: _renew_subs 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a641f2a20 con 0x7f4a64152e20 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:03.085 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 1 -- 10.20.193.6:0/1007802713 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400cd50 con 0x7f4a64152e20 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a3bfff640 10 monclient: dump: 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a617fa640 4 set_mon_vals no callback set 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 10 client.4261.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 10 monclient: _renew_subs 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a64159dc0 con 0x7f4a64152e20 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.075+0000 7f4a698bf640 1 librados: init done 2026-02-22T23:22:03.086 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.076+0000 7f4a698bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:03.087 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.076+0000 7f4a698bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:03.087 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.076+0000 7f4a698bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:22:03.087 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.076+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a640c6960 con 0x7f4a64152e20 2026-02-22T23:22:03.088 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.086+0000 7f4a3bfff640 1 -- 10.20.193.6:0/1007802713 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f4a54004020 con 0x7f4a64152e20 2026-02-22T23:22:03.089 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.087+0000 7f4a3bfff640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a34078d70 0x7f4a3407b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:03.089 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.087+0000 7f4a62ffd640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a34078d70 0x7f4a3407b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:03.089 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.087+0000 7f4a62ffd640 10 monclient: get_auth_request con 0x7f4a34078d70 auth_method 0 2026-02-22T23:22:03.090 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.088+0000 7f4a3bfff640 10 client.4261.objecter ms_handle_connect 0x7f4a34078d70 2026-02-22T23:22:03.090 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.088+0000 7f4a62ffd640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a34078d70 0x7f4a3407b230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4a58002790 tx=0x7f4a58033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:03.090 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.088+0000 7f4a3bfff640 1 -- 10.20.193.6:0/1007802713 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f4a5405dc40 con 0x7f4a64152e20 2026-02-22T23:22:03.090 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.088+0000 7f4a3bfff640 10 client.4261.objecter ms_dispatch2 0x7f4a64000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:03.090 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.088+0000 7f4a3bfff640 3 client.4261.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:03.091 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.088+0000 7f4a3bfff640 3 client.4261.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:03.091 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.088+0000 7f4a3bfff640 20 client.4261.objecter dump_active .. 0 homeless 2026-02-22T23:22:03.200 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.197+0000 7f4a3bfff640 1 -- 10.20.193.6:0/1007802713 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4a54061c00 con 0x7f4a64152e20 2026-02-22T23:22:03.200 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.197+0000 7f4a3bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:03.200 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.197+0000 7f4a3bfff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:03.287 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.285+0000 7f4a698bf640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:03.287 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.285+0000 7f4a698bf640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:03.288 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.285+0000 7f4a698bf640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:22:03.288 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.285+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a64156350 con 0x7f4a64152e20 2026-02-22T23:22:03.314 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.311+0000 7f4a3bfff640 1 -- 10.20.193.6:0/1007802713 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f4a54076e20 con 0x7f4a64152e20 2026-02-22T23:22:03.314 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.311+0000 7f4a3bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:03.314 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.312+0000 7f4a3bfff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:03.315 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:03.315 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 10 librados: watch_flush enter 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 10 librados: watch_flush exit 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 20 client.4261.objecter shutdown clearing up homeless session... 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 10 client.4261.objecter successfully canceled tick 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a34078d70 msgr2=0x7f4a3407b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a34078d70 0x7f4a3407b230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4a58002790 tx=0x7f4a58033000 comp rx=0 tx=0).stop 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 10 monclient: shutdown 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:03.318 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152e20 msgr2=0x7f4a6415ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152e20 0x7f4a6415ab80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e960 tx=0x7f4a5400ee30 comp rx=0 tx=0).stop 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 shutdown_connections 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a34078d70 0x7f4a3407b230 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4a64152e20 0x7f4a6415ab80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4a64152a40 0x7f4a641f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 --2- 10.20.193.6:0/1007802713 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4a6406dd40 0x7f4a641f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 >> 10.20.193.6:0/1007802713 conn(0x7f4a640c9f40 msgr2=0x7f4a640746c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 shutdown_connections 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 -- 10.20.193.6:0/1007802713 wait complete. 2026-02-22T23:22:03.319 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:03.314+0000 7f4a698bf640 1 librados: shutdown 2026-02-22T23:22:03.325 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:22:05.896 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:20.635 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:05.906 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:20.718 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:05.922 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:20.746 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:05.937 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:20.677 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:05.981 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:20.787 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:05.982 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:20.801 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.060 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:20.812 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.071 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:20.813 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.151 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:20.888 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.177 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:21.000 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.223 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:20.963 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.233 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:21.045 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.240 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:21.056 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.245 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:20.972 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.252 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:21.075 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.259 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:20.999 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.305 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:21.112 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.322 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:21.141 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.355 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:21.082 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.380 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:21.132 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.396 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:21.137 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.398 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:21.225 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.464 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:21.191 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.473 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:21.210 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.497 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:21.320 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.513 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:21.240 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.533 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:21.273 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.539 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:21.362 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.548 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:21.360 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.564 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:21.380 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.571 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:21.311 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.571 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:21.299 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.602 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:21.408 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.607 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:21.426 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.668 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:21.420 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.675 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:21.402 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.682 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:21.423 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.718 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:21.544 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.757 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:21.494 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.783 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:21.606 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.787 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:21.514 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.806 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:21.546 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.809 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:21.633 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.822 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:21.634 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.838 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:21.565 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.846 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:21.585 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.862 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:21.589 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.867 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:21.674 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.872 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:21.691 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.876 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:21.692 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.932 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:21.684 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.949 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:21.691 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:06.960 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:21.687 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.003 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:21.829 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.021 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:21.758 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.046 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:21.869 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.075 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:21.898 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.081 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:21.820 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.093 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:21.820 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.096 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:21.908 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.120 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:21.860 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.125 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:21.853 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.128 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:21.855 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.137 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:21.944 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.140 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:21.960 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.156 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:21.972 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.199 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:21.950 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.217 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:21.959 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.225 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:21.952 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.267 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:22.093 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.284 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:22.021 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.310 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:22.133 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.340 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:22.163 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.359 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:22.098 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.367 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:22.094 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.373 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:22.185 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.393 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:22.120 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.394 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:22.133 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.398 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:22.125 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.402 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:22.209 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.403 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:22.223 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.431 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:22.247 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.463 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:22.215 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.481 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:22.223 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.488 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:22.215 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.529 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:22.356 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.551 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:22.288 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.574 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:22.397 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.606 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:22.430 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.632 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:22.372 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.645 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:22.371 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.655 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:22.467 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.664 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:22.391 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.666 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:22.486 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.670 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:22.410 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.672 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:22.479 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.673 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:22.400 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.705 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:22.521 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.733 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:22.484 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.745 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:22.487 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.752 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:22.480 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.793 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:22.619 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.819 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:22.556 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.839 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:22.661 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.875 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:22.698 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.906 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:22.646 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.919 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:22.646 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.929 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:22.749 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.930 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:22.657 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.934 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:22.746 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.938 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:22.665 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.940 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:22.747 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.947 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:22.688 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:07.979 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:22.795 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.006 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:22.758 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.012 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:22.754 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.015 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:22.742 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.056 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:22.883 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.086 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:22.823 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.103 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:22.925 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.143 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:22.966 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.181 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:22.921 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.192 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:23.012 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.195 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:22.922 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.199 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:22.926 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.203 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:22.930 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.211 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:23.018 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.216 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:23.028 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.221 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:22.961 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.253 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:23.069 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.276 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:23.018 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.278 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:23.005 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.280 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:23.031 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.320 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:23.146 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.354 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:23.091 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.365 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:23.188 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.469 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:23.196 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.473 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:23.201 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.475 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:23.202 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.503 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:23.326 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.538 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:23.354 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.541 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:23.268 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.553 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:23.373 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.557 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:23.297 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.572 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:23.379 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.592 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:23.418 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.600 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:23.412 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.601 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:23.340 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.610 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:23.350 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.613 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:23.420 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.628 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:23.368 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.635 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:23.377 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.636 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:23.442 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.641 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:23.392 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.650 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:23.461 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.653 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:23.393 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.658 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:23.398 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.659 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:23.467 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.664 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:23.471 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.671 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:23.483 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.687 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:23.438 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.710 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:23.521 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.714 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:23.451 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.715 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:23.526 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.718 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:23.541 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.725 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:23.477 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.737 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:23.464 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.755 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:23.492 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.758 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:23.485 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.761 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:23.584 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.767 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:23.518 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.771 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:23.523 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.789 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:23.612 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.790 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:23.527 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.797 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:23.620 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.814 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:23.551 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.814 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:23.637 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.819 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:23.642 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.819 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:23.556 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.833 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:23.560 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.850 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:23.670 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.904 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:23.644 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.908 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:23.635 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.927 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:23.743 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.932 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:23.672 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.942 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:23.684 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.951 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:23.678 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.962 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:23.788 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.976 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:23.717 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:08.979 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:23.706 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.001 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:23.743 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.005 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:23.732 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.006 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:23.748 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.009 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:23.737 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.019 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:23.846 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.051 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:23.877 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.075 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:23.899 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.098 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:23.826 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.116 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:23.939 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.120 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:23.944 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.140 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:23.959 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.141 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:23.868 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.142 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:23.870 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.180 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:23.907 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.188 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:23.915 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.209 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:23.936 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.213 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:23.941 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.217 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:23.957 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.221 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:23.962 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.225 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:23.952 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.236 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:24.052 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.329 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:24.156 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.334 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:24.161 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.390 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:24.117 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.424 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:24.243 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.428 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:24.155 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.429 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:24.248 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.464 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:24.191 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.469 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:24.196 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.498 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:24.225 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.503 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:24.230 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.515 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:24.331 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.794 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:24.610 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:09.799 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:24.615 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:10.938 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:11.003 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.997+0000 7eff7321a640 10 monclient: get_monmap_and_config 2026-02-22T23:22:11.003 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.997+0000 7eff7321a640 10 monclient: build_initial_monmap 2026-02-22T23:22:11.003 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.997+0000 7eff7321a640 10 monclient: monmap: 2026-02-22T23:22:11.004 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:11.004 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:11.004 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:10.998201+0000 2026-02-22T23:22:11.004 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:10.998201+0000 2026-02-22T23:22:11.004 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:11.004 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:11.005 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:11.005 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:11.005 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:11.005 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:11.005 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:11.005 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:11.006 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:11.006 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 1 Processor -- start 2026-02-22T23:22:11.006 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 1 -- start start 2026-02-22T23:22:11.006 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient: init 2026-02-22T23:22:11.006 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:11.006 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient: _add_conns ranks=[0,1,4,3,2,5] 2026-02-22T23:22:11.007 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7eff6c1539a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:22:11.007 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7eff6c153d80 addr v1:10.20.193.40:6789/0 2026-02-22T23:22:11.007 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7eff6c0c8b00 0x7eff6c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:11.007 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient(hunting): picked mon.noname-b con 0x7eff6c0c8b00 addr v2:10.20.193.40:3300/0 2026-02-22T23:22:11.007 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff6c075070 con 0x7eff6c1539a0 2026-02-22T23:22:11.007 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff6c0c9490 con 0x7eff6c153d80 2026-02-22T23:22:11.008 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:11.008 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7eff6c0c5cd0 con 0x7eff6c0c8b00 2026-02-22T23:22:11.008 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:11.008 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.998+0000 7eff7321a640 10 monclient(hunting): authenticate will time out at 1005.725589s 2026-02-22T23:22:11.008 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.999+0000 7eff70f8f640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7eff6c1539a0 0x7eff6c0c7270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:51182/0 (socket says 10.20.193.6:51182) 2026-02-22T23:22:11.008 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.999+0000 7eff70f8f640 1 -- 10.20.193.6:0/2467546678 learned_addr learned my addr 10.20.193.6:0/2467546678 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:22:11.009 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.999+0000 7eff71790640 1 --2- 10.20.193.6:0/2467546678 >> v2:10.20.193.40:3300/0 conn(0x7eff6c0c8b00 0x7eff6c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:11.009 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.999+0000 7eff71790640 10 monclient(hunting): get_auth_request con 0x7eff6c0c8b00 auth_method 0 2026-02-22T23:22:11.009 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.999+0000 7eff71790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:11.009 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.999+0000 7eff71790640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:11.009 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:10.999+0000 7eff71790640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:11.009 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff71790640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:11.010 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff71790640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:11.010 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff71790640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:11.010 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff637fe640 1 -- 10.20.193.6:0/2467546678 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756377314 0 0) 0x7eff5c002920 con 0x7eff6c1539a0 2026-02-22T23:22:11.010 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff637fe640 10 monclient(hunting): _negotiate 2026-02-22T23:22:11.010 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff637fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:11.010 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:11.010 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff637fe640 10 monclient(hunting): my global_id is 4263 2026-02-22T23:22:11.011 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.000+0000 7eff637fe640 1 -- 10.20.193.6:0/2467546678 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff58003620 con 0x7eff6c1539a0 2026-02-22T23:22:11.011 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 10 monclient(hunting): handle_auth_done global_id 4264 payload 274 2026-02-22T23:22:11.011 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 1 -- 10.20.193.6:0/2467546678 >> v1:10.20.193.40:6789/0 conn(0x7eff6c153d80 legacy=0x7eff6c074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:11.011 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 1 -- 10.20.193.6:0/2467546678 >> v1:10.20.193.6:6789/0 conn(0x7eff6c1539a0 legacy=0x7eff6c0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:11.011 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 10 monclient: _finish_hunting 0 2026-02-22T23:22:11.011 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 1 monclient: found mon.noname-b 2026-02-22T23:22:11.012 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:11.012 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:22:11.012 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.001+0000 7eff71790640 1 -- 10.20.193.6:0/2467546678 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c0c5710 con 0x7eff6c0c8b00 2026-02-22T23:22:11.025 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.004+0000 7eff71790640 1 --2- 10.20.193.6:0/2467546678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 0x7eff6c0c8f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff68009d90 tx=0x7eff6802f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4493f81f8af9de1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:11.026 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 1 -- 10.20.193.6:0/2467546678 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6803c070 con 0x7eff6c0c8b00 2026-02-22T23:22:11.026 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:11.026 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:22:11.026 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: dump: 2026-02-22T23:22:11.026 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:11.027 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:11.027 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.027 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.027 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:11.027 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:11.027 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:11.028 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:11.028 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:11.028 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:11.028 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: _finish_auth 0 2026-02-22T23:22:11.028 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: _check_auth_tickets 2026-02-22T23:22:11.028 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:11.029 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 1 -- 10.20.193.6:0/2467546678 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff68037440 con 0x7eff6c0c8b00 2026-02-22T23:22:11.029 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:11.029 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 1 -- 10.20.193.6:0/2467546678 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68037810 con 0x7eff6c0c8b00 2026-02-22T23:22:11.029 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 5 monclient: authenticate success, global_id 4264 2026-02-22T23:22:11.029 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:11.029 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 4 set_mon_vals no callback set 2026-02-22T23:22:11.029 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:11.030 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:22:11.030 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff637fe640 10 monclient: dump: 2026-02-22T23:22:11.030 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:11.030 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:11.030 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.030 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.031 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:11.031 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:11.031 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:11.031 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:11.031 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:11.031 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:11.032 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 10 monclient: shutdown 2026-02-22T23:22:11.032 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:11.032 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 1 -- 10.20.193.6:0/2467546678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 msgr2=0x7eff6c0c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:11.032 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 1 --2- 10.20.193.6:0/2467546678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 0x7eff6c0c8f50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff68009d90 tx=0x7eff6802f1e0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.032 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 1 -- 10.20.193.6:0/2467546678 shutdown_connections 2026-02-22T23:22:11.032 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 1 --2- 10.20.193.6:0/2467546678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 0x7eff6c0c8f50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.033 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 1 -- 10.20.193.6:0/2467546678 >> 10.20.193.6:0/2467546678 conn(0x7eff6c0c9f40 msgr2=0x7eff6c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:11.033 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 1 -- 10.20.193.6:0/2467546678 shutdown_connections 2026-02-22T23:22:11.033 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.006+0000 7eff7321a640 1 -- 10.20.193.6:0/2467546678 wait complete. 2026-02-22T23:22:11.033 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient: build_initial_monmap 2026-02-22T23:22:11.033 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient: monmap: 2026-02-22T23:22:11.033 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:11.033 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:11.034 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:11.034 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:11.034 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:11.034 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:11.034 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:11.034 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:11.035 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:11.035 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:11.035 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 librados: starting msgr at 2026-02-22T23:22:11.035 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 librados: starting objecter 2026-02-22T23:22:11.035 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 Processor -- start 2026-02-22T23:22:11.035 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 -- start start 2026-02-22T23:22:11.035 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 librados: setting wanted keys 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 librados: calling monclient init 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient: init 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 0x7eff6c1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): picked mon.noname-b con 0x7eff6c0c8b00 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7eff6c1539a0 0x7eff6c14a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:11.036 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): picked mon.noname-c con 0x7eff6c1539a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:11.037 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7eff6c153d80 0x7eff6c14d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:11.037 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): picked mon.noname-a con 0x7eff6c153d80 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:11.037 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:11.037 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c0c8300 con 0x7eff6c153d80 2026-02-22T23:22:11.037 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:11.037 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c0c8020 con 0x7eff6c0c8b00 2026-02-22T23:22:11.037 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:11.038 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c0c85e0 con 0x7eff6c1539a0 2026-02-22T23:22:11.038 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:11.038 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.007+0000 7eff7321a640 10 monclient(hunting): authenticate will time out at 1005.734858s 2026-02-22T23:22:11.038 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7eff6c153d80 0x7eff6c14d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:11.038 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 0x7eff6c1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:11.038 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7eff6c153d80 0x7eff6c14d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41114/0 (socket says 10.20.193.6:41114) 2026-02-22T23:22:11.038 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 1 -- 10.20.193.6:0/274538564 learned_addr learned my addr 10.20.193.6:0/274538564 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 10 monclient(hunting): get_auth_request con 0x7eff6c153d80 auth_method 0 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 10 monclient(hunting): get_auth_request con 0x7eff6c0c8b00 auth_method 0 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff63fff640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7eff6c1539a0 0x7eff6c14a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:11.039 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff71790640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff70f8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff63fff640 10 monclient(hunting): get_auth_request con 0x7eff6c1539a0 auth_method 0 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff63fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff63fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:11.040 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.008+0000 7eff63fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.009+0000 7eff63fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.009+0000 7eff63fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.009+0000 7eff63fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 10 monclient(hunting): handle_auth_done global_id 4266 payload 824 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 1 -- 10.20.193.6:0/274538564 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7eff6c1539a0 msgr2=0x7eff6c14a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7eff6c1539a0 0x7eff6c14a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 1 -- 10.20.193.6:0/274538564 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 msgr2=0x7eff6c1f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 0x7eff6c1f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 10 monclient: _finish_hunting 0 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 1 monclient: found mon.noname-a 2026-02-22T23:22:11.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.010+0000 7eff71790640 1 -- 10.20.193.6:0/274538564 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c14ae20 con 0x7eff6c153d80 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.011+0000 7eff71790640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7eff6c153d80 0x7eff6c14d120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff6802f710 tx=0x7eff68004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.011+0000 7eff617fa640 10 client.?.objecter ms_handle_connect 0x7eff6c153d80 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.011+0000 7eff617fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 1 -- 10.20.193.6:0/274538564 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6803c050 con 0x7eff6c153d80 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: dump: 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.042 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: _finish_auth 0 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: _check_auth_tickets 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 1 -- 10.20.193.6:0/274538564 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff68048070 con 0x7eff6c153d80 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 1 -- 10.20.193.6:0/274538564 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68035a50 con 0x7eff6c153d80 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff617fa640 10 monclient: dump: 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:11.043 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff62ffd640 4 set_mon_vals no callback set 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 5 monclient: authenticate success, global_id 4266 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 monclient: _renew_subs 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c14b100 con 0x7eff6c153d80 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 client.4266.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 monclient: _renew_subs 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c14b6c0 con 0x7eff6c153d80 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 1 librados: init done 2026-02-22T23:22:11.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:11.045 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:11.045 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:11.045 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.013+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c0c52d0 con 0x7eff6c153d80 2026-02-22T23:22:11.121 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.120+0000 7eff617fa640 1 -- 10.20.193.6:0/274538564 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7eff68035bf0 con 0x7eff6c153d80 2026-02-22T23:22:11.122 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.120+0000 7eff617fa640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7eff3c078dc0 0x7eff3c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:11.122 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.120+0000 7eff70f8f640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7eff3c078dc0 0x7eff3c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:11.122 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.121+0000 7eff70f8f640 10 monclient: get_auth_request con 0x7eff3c078dc0 auth_method 0 2026-02-22T23:22:11.122 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.121+0000 7eff617fa640 10 client.4266.objecter ms_handle_connect 0x7eff3c078dc0 2026-02-22T23:22:11.123 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.121+0000 7eff70f8f640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7eff3c078dc0 0x7eff3c07b280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005980 tx=0x7eff5c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:11.130 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 1 -- 10.20.193.6:0/274538564 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7eff680845d0 con 0x7eff6c153d80 2026-02-22T23:22:11.130 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 10 client.4266.objecter ms_dispatch2 0x7eff6c000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:11.130 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 3 client.4266.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:11.130 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 3 client.4266.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:11.130 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 20 client.4266.objecter dump_active .. 0 homeless 2026-02-22T23:22:11.130 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 1 -- 10.20.193.6:0/274538564 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7eff68088590 con 0x7eff6c153d80 2026-02-22T23:22:11.131 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:11.131 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.128+0000 7eff617fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:11.218 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.216+0000 7eff7321a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:11.218 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.216+0000 7eff7321a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:11.218 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.216+0000 7eff7321a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:11.218 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.216+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff6c06b860 con 0x7eff6c153d80 2026-02-22T23:22:11.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.227+0000 7eff617fa640 1 -- 10.20.193.6:0/274538564 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7eff6808c890 con 0x7eff6c153d80 2026-02-22T23:22:11.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.227+0000 7eff617fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:11.228 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.227+0000 7eff617fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:11.229 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:11.229 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:11.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 10 librados: watch_flush enter 2026-02-22T23:22:11.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 10 librados: watch_flush exit 2026-02-22T23:22:11.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 20 client.4266.objecter shutdown clearing up homeless session... 2026-02-22T23:22:11.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 10 client.4266.objecter successfully canceled tick 2026-02-22T23:22:11.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7eff3c078dc0 msgr2=0x7eff3c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:11.231 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7eff3c078dc0 0x7eff3c07b280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005980 tx=0x7eff5c009040 comp rx=0 tx=0).stop 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 10 monclient: shutdown 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7eff6c153d80 msgr2=0x7eff6c14d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7eff6c153d80 0x7eff6c14d120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff6802f710 tx=0x7eff68004b40 comp rx=0 tx=0).stop 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 shutdown_connections 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7eff3c078dc0 0x7eff3c07b280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7eff6c153d80 0x7eff6c14d120 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7eff6c1539a0 0x7eff6c14a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 --2- 10.20.193.6:0/274538564 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7eff6c0c8b00 0x7eff6c1f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 >> 10.20.193.6:0/274538564 conn(0x7eff6c0c9f40 msgr2=0x7eff6c0cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 shutdown_connections 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 -- 10.20.193.6:0/274538564 wait complete. 2026-02-22T23:22:11.232 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:11.229+0000 7eff7321a640 1 librados: shutdown 2026-02-22T23:22:11.239 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:22:18.847 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:18.916 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.910+0000 7f526c908640 10 monclient: get_monmap_and_config 2026-02-22T23:22:18.916 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.910+0000 7f526c908640 10 monclient: build_initial_monmap 2026-02-22T23:22:18.916 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.910+0000 7f526c908640 10 monclient: monmap: 2026-02-22T23:22:18.917 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:18.917 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:18.917 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:18.911188+0000 2026-02-22T23:22:18.917 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:18.911188+0000 2026-02-22T23:22:18.917 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:18.917 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:18.918 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:18.918 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:18.918 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:18.918 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:18.918 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:18.918 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:18.919 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:18.919 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.910+0000 7f526c908640 1 Processor -- start 2026-02-22T23:22:18.919 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 1 -- start start 2026-02-22T23:22:18.919 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient: init 2026-02-22T23:22:18.919 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:18.919 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient: _add_conns ranks=[2,0,5,1,3,4] 2026-02-22T23:22:18.920 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5264152a40 addr v1:10.20.193.80:6789/0 2026-02-22T23:22:18.920 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5264152e20 addr v1:10.20.193.6:6789/0 2026-02-22T23:22:18.920 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f526406e450 0x7f52640c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:18.920 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient(hunting): picked mon.noname-c con 0x7f526406e450 addr v2:10.20.193.80:3300/0 2026-02-22T23:22:18.920 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52640c6d70 con 0x7f5264152e20 2026-02-22T23:22:18.920 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52640c7050 con 0x7f5264152a40 2026-02-22T23:22:18.921 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:18.921 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f52640c8550 con 0x7f526406e450 2026-02-22T23:22:18.921 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:18.921 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.911+0000 7f526c908640 10 monclient(hunting): authenticate will time out at 1013.638580s 2026-02-22T23:22:18.921 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.912+0000 7f5269e7c640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f5264152e20 0x7f526406dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:41278/0 (socket says 10.20.193.6:41278) 2026-02-22T23:22:18.921 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.912+0000 7f5269e7c640 1 -- 10.20.193.6:0/2966394547 learned_addr learned my addr 10.20.193.6:0/2966394547 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:22:18.922 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.912+0000 7f526ae7e640 1 --2- 10.20.193.6:0/2966394547 >> v2:10.20.193.80:3300/0 conn(0x7f526406e450 0x7f52640c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:18.922 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.912+0000 7f526ae7e640 10 monclient(hunting): get_auth_request con 0x7f526406e450 auth_method 0 2026-02-22T23:22:18.922 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.912+0000 7f526ae7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:18.922 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.912+0000 7f526ae7e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:18.922 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.912+0000 7f526ae7e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:18.922 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526ae7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:18.923 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526ae7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:18.923 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526ae7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:18.923 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526967b640 1 -- 10.20.193.6:0/2966394547 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593957221 0 0) 0x7f52640c6d70 con 0x7f5264152e20 2026-02-22T23:22:18.923 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526967b640 10 monclient(hunting): _negotiate 2026-02-22T23:22:18.923 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526967b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:18.923 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526967b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:18.924 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526967b640 10 monclient(hunting): my global_id is 4269 2026-02-22T23:22:18.924 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.913+0000 7f526967b640 1 -- 10.20.193.6:0/2966394547 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5254003620 con 0x7f5264152e20 2026-02-22T23:22:18.924 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 10 monclient(hunting): handle_auth_done global_id 4298 payload 274 2026-02-22T23:22:18.924 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 1 -- 10.20.193.6:0/2966394547 >> v1:10.20.193.80:6789/0 conn(0x7f5264152a40 legacy=0x7f52640c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:22:18.924 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 1 -- 10.20.193.6:0/2966394547 >> v1:10.20.193.6:6789/0 conn(0x7f5264152e20 legacy=0x7f526406dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:18.924 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 10 monclient: _finish_hunting 0 2026-02-22T23:22:18.925 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 1 monclient: found mon.noname-c 2026-02-22T23:22:18.925 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:18.925 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:22:18.925 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.914+0000 7f526ae7e640 1 -- 10.20.193.6:0/2966394547 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52640c7330 con 0x7f526406e450 2026-02-22T23:22:18.937 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.915+0000 7f526ae7e640 1 --2- 10.20.193.6:0/2966394547 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 0x7f52640c99a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5258009d90 tx=0x7f525802f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a941ca4597acebe server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:18.938 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 1 -- 10.20.193.6:0/2966394547 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525803c070 con 0x7f526406e450 2026-02-22T23:22:18.938 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:18.938 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:22:18.938 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: dump: 2026-02-22T23:22:18.938 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:18.938 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:18.939 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.939 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.939 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:18.939 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:18.939 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:18.939 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:18.940 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:18.940 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:18.940 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: _finish_auth 0 2026-02-22T23:22:18.940 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: _check_auth_tickets 2026-02-22T23:22:18.940 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:18.940 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 5 monclient: authenticate success, global_id 4298 2026-02-22T23:22:18.941 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:18.941 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 1 -- 10.20.193.6:0/2966394547 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5258037440 con 0x7f526406e450 2026-02-22T23:22:18.941 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:18.941 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 1 -- 10.20.193.6:0/2966394547 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580377f0 con 0x7f526406e450 2026-02-22T23:22:18.941 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:18.942 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 4 set_mon_vals no callback set 2026-02-22T23:22:18.942 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:18.942 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:22:18.942 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526967b640 10 monclient: dump: 2026-02-22T23:22:18.942 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:18.942 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:18.943 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.943 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.943 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:18.943 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:18.943 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:18.943 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:18.943 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:18.944 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:18.944 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 10 monclient: shutdown 2026-02-22T23:22:18.944 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:18.944 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 1 -- 10.20.193.6:0/2966394547 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 msgr2=0x7f52640c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:18.944 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 1 --2- 10.20.193.6:0/2966394547 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 0x7f52640c99a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5258009d90 tx=0x7f525802f1e0 comp rx=0 tx=0).stop 2026-02-22T23:22:18.944 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 1 -- 10.20.193.6:0/2966394547 shutdown_connections 2026-02-22T23:22:18.945 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 1 --2- 10.20.193.6:0/2966394547 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 0x7f52640c99a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:18.945 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 1 -- 10.20.193.6:0/2966394547 >> 10.20.193.6:0/2966394547 conn(0x7f52640c9f40 msgr2=0x7f52640ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:18.945 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 1 -- 10.20.193.6:0/2966394547 shutdown_connections 2026-02-22T23:22:18.945 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.918+0000 7f526c908640 1 -- 10.20.193.6:0/2966394547 wait complete. 2026-02-22T23:22:18.945 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 10 monclient: build_initial_monmap 2026-02-22T23:22:18.945 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 10 monclient: monmap: 2026-02-22T23:22:18.945 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:18.946 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 1 librados: starting msgr at 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 1 librados: starting objecter 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 1 Processor -- start 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 1 -- start start 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 1 librados: setting wanted keys 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 1 librados: calling monclient init 2026-02-22T23:22:18.947 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 10 monclient: init 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.919+0000 7f526c908640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 0x7f526414cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): picked mon.noname-c con 0x7f526406e450 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5264152a40 0x7f52641f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): picked mon.noname-a con 0x7f5264152a40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5264152e20 0x7f52641f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): picked mon.noname-b con 0x7f5264152e20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:18.948 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f526406d820 con 0x7f5264152a40 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f526406d5d0 con 0x7f5264152e20 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52640c6960 con 0x7f526406e450 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526c908640 10 monclient(hunting): authenticate will time out at 1013.647028s 2026-02-22T23:22:18.949 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5264152a40 0x7f52641f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 0x7f526414cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5264152a40 0x7f52641f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43050/0 (socket says 10.20.193.6:43050) 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 1 -- 10.20.193.6:0/904312561 learned_addr learned my addr 10.20.193.6:0/904312561 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 10 monclient(hunting): get_auth_request con 0x7f5264152a40 auth_method 0 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 10 monclient(hunting): get_auth_request con 0x7f526406e450 auth_method 0 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:18.950 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526ae7e640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5264152e20 0x7f52641f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f5269e7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526a67d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526ae7e640 10 monclient(hunting): get_auth_request con 0x7f5264152e20 auth_method 0 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526ae7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526ae7e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.920+0000 7f526ae7e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.921+0000 7f526ae7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.921+0000 7f526ae7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.921+0000 7f526ae7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:18.951 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 10 monclient(hunting): handle_auth_done global_id 4272 payload 824 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 1 -- 10.20.193.6:0/904312561 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 msgr2=0x7f526414cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 0x7f526414cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 1 -- 10.20.193.6:0/904312561 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5264152e20 msgr2=0x7f52641f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5264152e20 0x7f52641f2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 10 monclient: _finish_hunting 0 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 1 monclient: found mon.noname-a 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.922+0000 7f5269e7c640 1 -- 10.20.193.6:0/904312561 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526414e550 con 0x7f5264152a40 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.923+0000 7f5269e7c640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5264152a40 0x7f52641f1c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f525000c550 tx=0x7f525000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.923+0000 7f524effd640 10 client.?.objecter ms_handle_connect 0x7f5264152a40 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.923+0000 7f524effd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 1 -- 10.20.193.6:0/904312561 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250017070 con 0x7f5264152a40 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:18.952 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: dump: 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: _finish_auth 0 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: _check_auth_tickets 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 1 -- 10.20.193.6:0/904312561 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5250005de0 con 0x7f5264152a40 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:18.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 1 -- 10.20.193.6:0/904312561 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250007270 con 0x7f5264152a40 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 5 monclient: authenticate success, global_id 4272 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 10 monclient: _renew_subs 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52641f05a0 con 0x7f5264152a40 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f5268e7a640 4 set_mon_vals no callback set 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f524effd640 10 monclient: dump: 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:18.954 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 10 client.4272.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 10 monclient: _renew_subs 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52641f0e70 con 0x7f5264152a40 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.925+0000 7f526c908640 1 librados: init done 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.926+0000 7f526c908640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.926+0000 7f526c908640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.926+0000 7f526c908640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:18.955 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.926+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526415cc40 con 0x7f5264152a40 2026-02-22T23:22:18.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.938+0000 7f524effd640 1 -- 10.20.193.6:0/904312561 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f5250007410 con 0x7f5264152a40 2026-02-22T23:22:18.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.938+0000 7f524effd640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5234078dc0 0x7f523407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:18.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.938+0000 7f526a67d640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5234078dc0 0x7f523407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:18.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.938+0000 7f526a67d640 10 monclient: get_auth_request con 0x7f5234078dc0 auth_method 0 2026-02-22T23:22:18.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.939+0000 7f524effd640 10 client.4272.objecter ms_handle_connect 0x7f5234078dc0 2026-02-22T23:22:18.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.939+0000 7f526a67d640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5234078dc0 0x7f523407b280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f525c005160 tx=0x7f525c007f30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:18.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.940+0000 7f524effd640 1 -- 10.20.193.6:0/904312561 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f5250007730 con 0x7f5264152a40 2026-02-22T23:22:18.957 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.940+0000 7f524effd640 10 client.4272.objecter ms_dispatch2 0x7f5264000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:18.957 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.940+0000 7f524effd640 3 client.4272.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:18.957 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.940+0000 7f524effd640 3 client.4272.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:18.957 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:18.940+0000 7f524effd640 20 client.4272.objecter dump_active .. 0 homeless 2026-02-22T23:22:19.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.042+0000 7f524effd640 1 -- 10.20.193.6:0/904312561 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5250022080 con 0x7f5264152a40 2026-02-22T23:22:19.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.042+0000 7f524effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:19.044 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.042+0000 7f524effd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:19.135 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.134+0000 7f526c908640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:19.136 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.134+0000 7f526c908640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:19.136 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.134+0000 7f526c908640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:19.136 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.134+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5264156350 con 0x7f5264152a40 2026-02-22T23:22:19.146 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.144+0000 7f524effd640 1 -- 10.20.193.6:0/904312561 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f52500611c0 con 0x7f5264152a40 2026-02-22T23:22:19.146 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.144+0000 7f524effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:19.146 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.144+0000 7f524effd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:19.146 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:19.146 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 10 librados: watch_flush enter 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 10 librados: watch_flush exit 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 20 client.4272.objecter shutdown clearing up homeless session... 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 10 client.4272.objecter successfully canceled tick 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5234078dc0 msgr2=0x7f523407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5234078dc0 0x7f523407b280 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f525c005160 tx=0x7f525c007f30 comp rx=0 tx=0).stop 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 10 monclient: shutdown 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:19.149 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5264152a40 msgr2=0x7f52641f1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5264152a40 0x7f52641f1c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f525000c550 tx=0x7f525000ca20 comp rx=0 tx=0).stop 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 shutdown_connections 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5234078dc0 0x7f523407b280 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5264152e20 0x7f52641f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5264152a40 0x7f52641f1c30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 --2- 10.20.193.6:0/904312561 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f526406e450 0x7f526414cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 >> 10.20.193.6:0/904312561 conn(0x7f52640c9f40 msgr2=0x7f5264150d00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 shutdown_connections 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 -- 10.20.193.6:0/904312561 wait complete. 2026-02-22T23:22:19.150 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:19.147+0000 7f526c908640 1 librados: shutdown 2026-02-22T23:22:19.157 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:22:20.227 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:34.967 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.398 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:35.205 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.404 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:35.141 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.508 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:35.320 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.521 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:35.272 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.538 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:35.280 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.580 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:35.403 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.827 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:35.567 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.863 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:35.591 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:20.907 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:35.731 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:21.044 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:35.771 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:21.127 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:35.953 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:21.245 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:36.065 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:21.314 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:36.041 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:21.395 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:36.122 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:21.516 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:36.332 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.573 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:37.312 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.621 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:37.361 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.624 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:37.364 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.791 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:37.528 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.821 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:37.628 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.835 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:37.572 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.838 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:37.575 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.852 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:37.660 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.854 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:37.606 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.855 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:37.663 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.860 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:37.672 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.860 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:37.602 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.872 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:37.695 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.885 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:37.627 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.888 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:37.630 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.911 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:37.733 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:22.914 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:37.737 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.138 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:37.865 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.141 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:37.892 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.143 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:37.895 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.149 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:37.961 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.152 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:37.964 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.165 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:37.892 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.168 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:37.895 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.202 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:37.942 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.234 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:38.057 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.242 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:37.969 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.243 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:37.983 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.246 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:37.986 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.264 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:37.991 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.265 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:38.088 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.267 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:37.994 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.268 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:38.091 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.331 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:38.150 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.450 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:38.276 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.621 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:38.440 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.624 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:38.443 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.641 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:38.368 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.735 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:38.561 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.738 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:38.564 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.739 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:38.466 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.852 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:38.668 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.884 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:38.700 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.887 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:38.703 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.923 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:38.650 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:23.926 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:38.653 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.019 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:38.745 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.022 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:38.748 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.427 INFO:tasks.ceph.osd.6.trial080.stderr:==00:00:00:39.167 44713== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.499 INFO:tasks.ceph.osd.9.trial040.stderr:==00:00:00:39.306 44611== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.528 INFO:tasks.ceph.osd.2.trial080.stderr:==00:00:00:39.270 44711== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.704 INFO:tasks.ceph.osd.14.trial080.stderr:==00:00:00:39.440 44714== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.730 INFO:tasks.ceph.osd.8.trial006.stderr:==00:00:00:39.552 45052== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.804 INFO:tasks.ceph.osd.13.trial040.stderr:==00:00:00:39.556 44638== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.898 INFO:tasks.ceph.osd.10.trial080.stderr:==00:00:00:39.637 44712== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.903 INFO:tasks.ceph.osd.5.trial040.stderr:==00:00:00:39.715 44610== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:24.979 INFO:tasks.ceph.osd.4.trial006.stderr:==00:00:00:39.802 45051== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:25.015 INFO:tasks.ceph.osd.11.trial189.stderr:==00:00:00:39.742 44907== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:25.025 INFO:tasks.ceph.osd.7.trial189.stderr:==00:00:00:39.752 44906== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:25.437 INFO:tasks.ceph.osd.12.trial006.stderr:==00:00:00:40.256 45053== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:25.544 INFO:tasks.ceph.osd.1.trial040.stderr:==00:00:00:40.360 44609== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:25.557 INFO:tasks.ceph.osd.0.trial006.stderr:==00:00:00:40.383 45050== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:25.661 INFO:tasks.ceph.osd.3.trial189.stderr:==00:00:00:40.388 44905== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:25.785 INFO:tasks.ceph.osd.15.trial189.stderr:==00:00:00:40.513 44908== Warning: unimplemented fcntl command: 1038 2026-02-22T23:22:26.766 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:26.826 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.824+0000 7f612e06d640 10 monclient: get_monmap_and_config 2026-02-22T23:22:26.826 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.824+0000 7f612e06d640 10 monclient: build_initial_monmap 2026-02-22T23:22:26.827 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.824+0000 7f612e06d640 10 monclient: monmap: 2026-02-22T23:22:26.827 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:26.827 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:26.827 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:26.825128+0000 2026-02-22T23:22:26.827 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:26.825128+0000 2026-02-22T23:22:26.827 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:26.828 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:26.828 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:26.828 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:26.828 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:26.828 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:26.828 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:26.829 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:26.829 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:26.829 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.824+0000 7f612e06d640 1 Processor -- start 2026-02-22T23:22:26.829 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 1 -- start start 2026-02-22T23:22:26.829 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient: init 2026-02-22T23:22:26.839 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:26.839 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient: _add_conns ranks=[5,3,2,1,4,0] 2026-02-22T23:22:26.839 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f61281561d0 0x7f612806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient(hunting): picked mon.noname-c con 0x7f61281561d0 addr v2:10.20.193.80:3300/0 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f61280c9b60 0x7f612815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient(hunting): picked mon.noname-a con 0x7f61280c9b60 addr v2:10.20.193.6:3300/0 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f61281565b0 addr v1:10.20.193.80:6789/0 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f612806c8f0 con 0x7f61281565b0 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6128074c00 con 0x7f61280c9b60 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f612806d7e0 con 0x7f61281561d0 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.825+0000 7f612e06d640 10 monclient(hunting): authenticate will time out at 1021.552690s 2026-02-22T23:22:26.840 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f61280c9b60 0x7f612815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f61280c9b60 0x7f612815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43060/0 (socket says 10.20.193.6:43060) 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 1 -- 10.20.193.6:0/482054935 learned_addr learned my addr 10.20.193.6:0/482054935 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 10 monclient(hunting): get_auth_request con 0x7f61280c9b60 auth_method 0 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f61277fe640 1 --2- 10.20.193.6:0/482054935 >> v2:10.20.193.80:3300/0 conn(0x7f61281561d0 0x7f612806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f6126ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:26.841 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f61277fe640 10 monclient(hunting): get_auth_request con 0x7f61281561d0 auth_method 0 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f61277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f61277fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.826+0000 7f61277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f61277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f61277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f61277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 10 monclient(hunting): handle_auth_done global_id 4275 payload 274 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 1 -- 10.20.193.6:0/482054935 >> v2:10.20.193.80:3300/0 conn(0x7f61281561d0 msgr2=0x7f612806dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 1 --2- 10.20.193.6:0/482054935 >> v2:10.20.193.80:3300/0 conn(0x7f61281561d0 0x7f612806dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:26.842 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 1 -- 10.20.193.6:0/482054935 >> v1:10.20.193.80:6789/0 conn(0x7f61281565b0 legacy=0x7f612815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 1 monclient: found mon.noname-a 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.827+0000 7f6126ffd640 1 -- 10.20.193.6:0/482054935 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612806cbd0 con 0x7f61280c9b60 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.828+0000 7f6126ffd640 1 --2- 10.20.193.6:0/482054935 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61280c9b60 0x7f612815d4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f61280cd410 tx=0x7f611802ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9345855325546599 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 1 -- 10.20.193.6:0/482054935 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611803c070 con 0x7f61280c9b60 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:22:26.843 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: dump: 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: _finish_auth 0 2026-02-22T23:22:26.844 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: _check_auth_tickets 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 1 -- 10.20.193.6:0/482054935 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611802fc70 con 0x7f61280c9b60 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 5 monclient: authenticate success, global_id 4275 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 1 -- 10.20.193.6:0/482054935 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118037660 con 0x7f61280c9b60 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 4 set_mon_vals no callback set 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f61267fc640 10 monclient: dump: 2026-02-22T23:22:26.845 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 10 monclient: shutdown 2026-02-22T23:22:26.846 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 -- 10.20.193.6:0/482054935 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61280c9b60 msgr2=0x7f612815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 --2- 10.20.193.6:0/482054935 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61280c9b60 0x7f612815d4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f61280cd410 tx=0x7f611802ef60 comp rx=0 tx=0).stop 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 -- 10.20.193.6:0/482054935 shutdown_connections 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 --2- 10.20.193.6:0/482054935 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61280c9b60 0x7f612815d4a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 --2- 10.20.193.6:0/482054935 >> v2:10.20.193.80:3300/0 conn(0x7f61281561d0 0x7f612806dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 -- 10.20.193.6:0/482054935 >> 10.20.193.6:0/482054935 conn(0x7f61280cc350 msgr2=0x7f61280cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 -- 10.20.193.6:0/482054935 shutdown_connections 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.830+0000 7f612e06d640 1 -- 10.20.193.6:0/482054935 wait complete. 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 10 monclient: build_initial_monmap 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 10 monclient: monmap: 2026-02-22T23:22:26.847 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 1 librados: starting msgr at 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 1 librados: starting objecter 2026-02-22T23:22:26.848 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 1 Processor -- start 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 1 -- start start 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 1 librados: setting wanted keys 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 1 librados: calling monclient init 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.831+0000 7f612e06d640 10 monclient: init 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f61280c9b60 0x7f612815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): picked mon.noname-b con 0x7f61280c9b60 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f61281561d0 0x7f612814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): picked mon.noname-c con 0x7f61281561d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61281f76f0 0x7f6128154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:26.849 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): picked mon.noname-a con 0x7f61281f76f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f612806d470 con 0x7f61281f76f0 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f612806d190 con 0x7f61280c9b60 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6128069250 con 0x7f61281561d0 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f612e06d640 10 monclient(hunting): authenticate will time out at 1021.559391s 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6127fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61281f76f0 0x7f6128154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6127fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61281f76f0 0x7f6128154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43076/0 (socket says 10.20.193.6:43076) 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6127fff640 1 -- 10.20.193.6:0/2525764987 learned_addr learned my addr 10.20.193.6:0/2525764987 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6127fff640 10 monclient(hunting): get_auth_request con 0x7f61281f76f0 auth_method 0 2026-02-22T23:22:26.850 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6127fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6127fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6127fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.832+0000 7f6126ffd640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f61281561d0 0x7f612814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6127fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6127fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6127fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f61280c9b60 0x7f612815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6126ffd640 10 monclient(hunting): get_auth_request con 0x7f61281561d0 auth_method 0 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6126ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6126ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6126ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:26.851 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 10 monclient(hunting): get_auth_request con 0x7f61280c9b60 auth_method 0 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6126ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6126ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f6126ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.833+0000 7f61277fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 10 monclient(hunting): handle_auth_done global_id 4278 payload 824 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 1 -- 10.20.193.6:0/2525764987 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f61281561d0 msgr2=0x7f612814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:26.852 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f61281561d0 0x7f612814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 1 -- 10.20.193.6:0/2525764987 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f61280c9b60 msgr2=0x7f612815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f61280c9b60 0x7f612815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 10 monclient: _finish_hunting 0 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 1 monclient: found mon.noname-a 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.834+0000 7f6127fff640 1 -- 10.20.193.6:0/2525764987 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61280ce2b0 con 0x7f61281f76f0 2026-02-22T23:22:26.853 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.835+0000 7f6127fff640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61281f76f0 0x7f6128154180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f611c00cba0 tx=0x7f611c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.835+0000 7f6103fff640 10 client.?.objecter ms_handle_connect 0x7f61281f76f0 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.835+0000 7f6103fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 1 -- 10.20.193.6:0/2525764987 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c00ad50 con 0x7f61281f76f0 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: dump: 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:26.854 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: _finish_auth 0 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: _check_auth_tickets 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 1 -- 10.20.193.6:0/2525764987 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c003a70 con 0x7f61281f76f0 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 1 -- 10.20.193.6:0/2525764987 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0050d0 con 0x7f61281f76f0 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:26.855 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6103fff640 10 monclient: dump: 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:26.856 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f6125ffb640 4 set_mon_vals no callback set 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 5 monclient: authenticate success, global_id 4278 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 monclient: _renew_subs 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61280ce590 con 0x7f61281f76f0 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 client.4278.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 monclient: _renew_subs 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61280ceb50 con 0x7f61281f76f0 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 1 librados: init done 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:26.857 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:26.858 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:26.858 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.837+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6128076e10 con 0x7f61281f76f0 2026-02-22T23:22:26.859 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.850+0000 7f6103fff640 1 -- 10.20.193.6:0/2525764987 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f611c005270 con 0x7f61281f76f0 2026-02-22T23:22:26.859 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.850+0000 7f6103fff640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60f8078df0 0x7f60f807b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:26.859 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.850+0000 7f61277fe640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60f8078df0 0x7f60f807b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:26.859 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.850+0000 7f61277fe640 10 monclient: get_auth_request con 0x7f60f8078df0 auth_method 0 2026-02-22T23:22:26.859 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.851+0000 7f6103fff640 10 client.4278.objecter ms_handle_connect 0x7f60f8078df0 2026-02-22T23:22:26.859 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.851+0000 7f61277fe640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60f8078df0 0x7f60f807b2b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c40 tx=0x7f6114009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:26.953 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 1 -- 10.20.193.6:0/2525764987 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f611c05e610 con 0x7f61281f76f0 2026-02-22T23:22:26.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 10 client.4278.objecter ms_dispatch2 0x7f6128000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:26.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 3 client.4278.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:26.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 3 client.4278.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:26.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 20 client.4278.objecter dump_active .. 0 homeless 2026-02-22T23:22:26.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 1 -- 10.20.193.6:0/2525764987 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6128076e10 con 0x7f61281f76f0 2026-02-22T23:22:26.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:26.954 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:26.952+0000 7f6103fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:27.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.039+0000 7f612e06d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:27.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.039+0000 7f612e06d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:27.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.039+0000 7f612e06d640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:27.041 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.039+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61280cce70 con 0x7f61281f76f0 2026-02-22T23:22:27.051 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.049+0000 7f6103fff640 1 -- 10.20.193.6:0/2525764987 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f611c0625d0 con 0x7f61281f76f0 2026-02-22T23:22:27.051 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.049+0000 7f6103fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:27.051 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.049+0000 7f6103fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:27.051 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:27.052 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:27.054 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 10 librados: watch_flush enter 2026-02-22T23:22:27.054 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 10 librados: watch_flush exit 2026-02-22T23:22:27.054 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 20 client.4278.objecter shutdown clearing up homeless session... 2026-02-22T23:22:27.054 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 10 client.4278.objecter successfully canceled tick 2026-02-22T23:22:27.054 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60f8078df0 msgr2=0x7f60f807b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:27.054 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60f8078df0 0x7f60f807b2b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6114009c40 tx=0x7f6114009290 comp rx=0 tx=0).stop 2026-02-22T23:22:27.054 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 10 monclient: shutdown 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61281f76f0 msgr2=0x7f6128154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.051+0000 7f612e06d640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61281f76f0 0x7f6128154180 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f611c00cba0 tx=0x7f611c00a430 comp rx=0 tx=0).stop 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 shutdown_connections 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60f8078df0 0x7f60f807b2b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f61281f76f0 0x7f6128154180 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f61281561d0 0x7f612814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 --2- 10.20.193.6:0/2525764987 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f61280c9b60 0x7f612815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 >> 10.20.193.6:0/2525764987 conn(0x7f61280cc350 msgr2=0x7f6128150980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:27.055 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 shutdown_connections 2026-02-22T23:22:27.056 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 -- 10.20.193.6:0/2525764987 wait complete. 2026-02-22T23:22:27.056 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:27.052+0000 7f612e06d640 1 librados: shutdown 2026-02-22T23:22:27.061 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:22:32.721 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:16.970 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.774 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:17.019 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.777 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:17.027 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.802 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:17.013 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.804 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:17.051 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.832 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:17.043 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.894 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:17.144 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.921 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:17.167 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:32.952 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:17.163 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:34.673 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:34.733 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.731+0000 7f3dde1d5640 10 monclient: get_monmap_and_config 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.731+0000 7f3dde1d5640 10 monclient: build_initial_monmap 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.731+0000 7f3dde1d5640 10 monclient: monmap: 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:34.732508+0000 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:34.732508+0000 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:34.734 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 1 Processor -- start 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 1 -- start start 2026-02-22T23:22:34.735 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient: init 2026-02-22T23:22:34.737 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:34.737 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient: _add_conns ranks=[1,5,2,3,0,4] 2026-02-22T23:22:34.737 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3dd81539a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f3dd8153d80 0x7f3dd8074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient(hunting): picked mon.noname-c con 0x7f3dd8153d80 addr v2:10.20.193.80:3300/0 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3dd80c8b00 addr v1:10.20.193.80:6789/0 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd80c5710 con 0x7f3dd81539a0 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd80c59f0 con 0x7f3dd80c8b00 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd80c5cd0 con 0x7f3dd8153d80 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:34.738 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.732+0000 7f3dde1d5640 10 monclient(hunting): authenticate will time out at 1029.459892s 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.733+0000 7f3dd6ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f3dd8153d80 0x7f3dd8074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.733+0000 7f3dd77fe640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f3dd81539a0 0x7f3dd80c7270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:44494/0 (socket says 10.20.193.6:44494) 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.733+0000 7f3dd77fe640 1 -- 10.20.193.6:0/198244269 learned_addr learned my addr 10.20.193.6:0/198244269 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.733+0000 7f3dd6ffd640 10 monclient(hunting): get_auth_request con 0x7f3dd8153d80 auth_method 0 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.733+0000 7f3dd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.733+0000 7f3dd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.733+0000 7f3dd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.734+0000 7f3dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.734+0000 7f3dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.734+0000 7f3dd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 10 monclient(hunting): handle_auth_done global_id 4313 payload 274 2026-02-22T23:22:34.739 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 1 -- 10.20.193.6:0/198244269 >> v1:10.20.193.80:6789/0 conn(0x7f3dd80c8b00 legacy=0x7f3dd80c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:34.740 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 1 -- 10.20.193.6:0/198244269 >> v1:10.20.193.40:6789/0 conn(0x7f3dd81539a0 legacy=0x7f3dd80c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:34.740 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:22:34.740 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 1 monclient: found mon.noname-c 2026-02-22T23:22:34.740 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:34.740 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:22:34.740 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.735+0000 7f3dd6ffd640 1 -- 10.20.193.6:0/198244269 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd80c8020 con 0x7f3dd8153d80 2026-02-22T23:22:34.740 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.737+0000 7f3dd6ffd640 1 --2- 10.20.193.6:0/198244269 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd8153d80 0x7f3dd8074960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80098a0 tx=0x7f3dc802edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0682af6cc34f5c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:34.745 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 1 -- 10.20.193.6:0/198244269 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc803c070 con 0x7f3dd8153d80 2026-02-22T23:22:34.745 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: dump: 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:34.746 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: _finish_auth 0 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 1 -- 10.20.193.6:0/198244269 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc802fae0 con 0x7f3dd8153d80 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 5 monclient: authenticate success, global_id 4313 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 1 -- 10.20.193.6:0/198244269 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8037440 con 0x7f3dd8153d80 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:34.747 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 4 set_mon_vals no callback set 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dd67fc640 10 monclient: dump: 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:34.748 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 10 monclient: shutdown 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 1 -- 10.20.193.6:0/198244269 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd8153d80 msgr2=0x7f3dd8074960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/198244269 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd8153d80 0x7f3dd8074960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80098a0 tx=0x7f3dc802edd0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 1 -- 10.20.193.6:0/198244269 shutdown_connections 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/198244269 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd8153d80 0x7f3dd8074960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 1 -- 10.20.193.6:0/198244269 >> 10.20.193.6:0/198244269 conn(0x7f3dd80c9f40 msgr2=0x7f3dd80ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 1 -- 10.20.193.6:0/198244269 shutdown_connections 2026-02-22T23:22:34.749 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.740+0000 7f3dde1d5640 1 -- 10.20.193.6:0/198244269 wait complete. 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient: build_initial_monmap 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient: monmap: 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:34.750 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 librados: starting msgr at 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 librados: starting objecter 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 Processor -- start 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 -- start start 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 librados: setting wanted keys 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 librados: calling monclient init 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient: init 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd80c8b00 0x7f3dd814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): picked mon.noname-c con 0x7f3dd80c8b00 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:34.751 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3dd81539a0 0x7f3dd814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): picked mon.noname-b con 0x7f3dd81539a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3dd8153d80 0x7f3dd81f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): picked mon.noname-a con 0x7f3dd8153d80 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80c52d0 con 0x7f3dd8153d80 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80c85e0 con 0x7f3dd81539a0 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80c5450 con 0x7f3dd80c8b00 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:34.752 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.741+0000 7f3dde1d5640 10 monclient(hunting): authenticate will time out at 1029.468999s 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd80c8b00 0x7f3dd814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd6ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3dd81539a0 0x7f3dd814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd7fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3dd8153d80 0x7f3dd81f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd80c8b00 0x7f3dd814d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:41834/0 (socket says 10.20.193.6:41834) 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 1 -- 10.20.193.6:0/466931473 learned_addr learned my addr 10.20.193.6:0/466931473 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 10 monclient(hunting): get_auth_request con 0x7f3dd80c8b00 auth_method 0 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd6ffd640 10 monclient(hunting): get_auth_request con 0x7f3dd81539a0 auth_method 0 2026-02-22T23:22:34.753 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd7fff640 10 monclient(hunting): get_auth_request con 0x7f3dd8153d80 auth_method 0 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.742+0000 7f3dd7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.743+0000 7f3dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.743+0000 7f3dd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:34.754 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.743+0000 7f3dd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.743+0000 7f3dd7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.743+0000 7f3dd7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.743+0000 7f3dd7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 10 monclient(hunting): handle_auth_done global_id 4319 payload 824 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 1 -- 10.20.193.6:0/466931473 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3dd81539a0 msgr2=0x7f3dd814b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3dd81539a0 0x7f3dd814b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 1 -- 10.20.193.6:0/466931473 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3dd8153d80 msgr2=0x7f3dd81f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3dd8153d80 0x7f3dd81f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 10 monclient: _finish_hunting 0 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 1 monclient: found mon.noname-c 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:22:34.755 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.744+0000 7f3dd77fe640 1 -- 10.20.193.6:0/466931473 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd814ba10 con 0x7f3dd80c8b00 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.745+0000 7f3dd77fe640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd80c8b00 0x7f3dd814d120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c760 tx=0x7f3dc400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.745+0000 7f3db3fff640 10 client.?.objecter ms_handle_connect 0x7f3dd80c8b00 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.745+0000 7f3db3fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 1 -- 10.20.193.6:0/466931473 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4017070 con 0x7f3dd80c8b00 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: dump: 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:34.758 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: _finish_auth 0 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 1 -- 10.20.193.6:0/466931473 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4005b90 con 0x7f3dd80c8b00 2026-02-22T23:22:34.759 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 5 monclient: authenticate success, global_id 4319 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 monclient: _renew_subs 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd806d610 con 0x7f3dd80c8b00 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 1 -- 10.20.193.6:0/466931473 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4007070 con 0x7f3dd80c8b00 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3db3fff640 10 monclient: dump: 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:34.760 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dd5ffb640 4 set_mon_vals no callback set 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 client.4319.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 monclient: _renew_subs 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:34.761 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd806dbd0 con 0x7f3dd80c8b00 2026-02-22T23:22:34.762 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 1 librados: init done 2026-02-22T23:22:34.762 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:34.762 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:34.762 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:34.762 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.747+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd80c8f50 con 0x7f3dd80c8b00 2026-02-22T23:22:34.763 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.759+0000 7f3db3fff640 1 -- 10.20.193.6:0/466931473 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f3dc4007210 con 0x7f3dd80c8b00 2026-02-22T23:22:34.763 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.759+0000 7f3db3fff640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3da8078d70 0x7f3da807b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:34.763 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.760+0000 7f3dd6ffd640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3da8078d70 0x7f3da807b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:34.763 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.760+0000 7f3dd6ffd640 10 monclient: get_auth_request con 0x7f3da8078d70 auth_method 0 2026-02-22T23:22:34.763 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.760+0000 7f3dd6ffd640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3da8078d70 0x7f3da807b230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8039000 tx=0x7f3dc8033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:34.763 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.760+0000 7f3db3fff640 10 client.4319.objecter ms_handle_connect 0x7f3da8078d70 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 1 -- 10.20.193.6:0/466931473 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f3dc405d480 con 0x7f3dd80c8b00 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 10 client.4319.objecter ms_dispatch2 0x7f3dd8000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 3 client.4319.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 3 client.4319.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 20 client.4319.objecter dump_active .. 0 homeless 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 1 -- 10.20.193.6:0/466931473 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3dc4021020 con 0x7f3dd80c8b00 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:34.869 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.867+0000 7f3db3fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:34.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.955+0000 7f3dde1d5640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:34.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.955+0000 7f3dde1d5640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:34.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.955+0000 7f3dde1d5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:34.956 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.955+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dd80cd2a0 con 0x7f3dd80c8b00 2026-02-22T23:22:34.967 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3db3fff640 1 -- 10.20.193.6:0/466931473 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f3dc4061440 con 0x7f3dd80c8b00 2026-02-22T23:22:34.967 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3db3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:34.967 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3db3fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:34.967 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:34.967 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 10 librados: watch_flush enter 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 10 librados: watch_flush exit 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 20 client.4319.objecter shutdown clearing up homeless session... 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 10 client.4319.objecter successfully canceled tick 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3da8078d70 msgr2=0x7f3da807b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3da8078d70 0x7f3da807b230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8039000 tx=0x7f3dc8033000 comp rx=0 tx=0).stop 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 10 monclient: shutdown 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd80c8b00 msgr2=0x7f3dd814d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd80c8b00 0x7f3dd814d120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c760 tx=0x7f3dc400cc30 comp rx=0 tx=0).stop 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 shutdown_connections 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3da8078d70 0x7f3da807b230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3dd8153d80 0x7f3dd81f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.966+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3dd81539a0 0x7f3dd814b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.967+0000 7f3dde1d5640 1 --2- 10.20.193.6:0/466931473 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3dd80c8b00 0x7f3dd814d120 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.967+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 >> 10.20.193.6:0/466931473 conn(0x7f3dd80c9f40 msgr2=0x7f3dd814dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.967+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 shutdown_connections 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.967+0000 7f3dde1d5640 1 -- 10.20.193.6:0/466931473 wait complete. 2026-02-22T23:22:34.968 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:34.967+0000 7f3dde1d5640 1 librados: shutdown 2026-02-22T23:22:34.978 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:22:42.586 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.647+0000 7f8bea632640 10 monclient: get_monmap_and_config 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.647+0000 7f8bea632640 10 monclient: build_initial_monmap 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.647+0000 7f8bea632640 10 monclient: monmap: 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:42.649110+0000 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:42.649110+0000 2026-02-22T23:22:42.649 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.647+0000 7f8bea632640 1 Processor -- start 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 1 -- start start 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient: init 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient: _add_conns ranks=[4,2,3,0,5,1] 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8be41539a0 0x7f8be40c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient(hunting): picked mon.noname-b con 0x7f8be41539a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8be4153d80 addr v1:10.20.193.80:6789/0 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8be40c8b00 0x7f8be40c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:42.650 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient(hunting): picked mon.noname-a con 0x7f8be40c8b00 addr v2:10.20.193.6:3300/0 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be4075070 con 0x7f8be4153d80 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8be40c5cd0 con 0x7f8be40c8b00 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8be40c9490 con 0x7f8be41539a0 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.648+0000 7f8bea632640 10 monclient(hunting): authenticate will time out at 1037.376571s 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be37fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f8be4153d80 0x7f8be4074960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:49068/0 (socket says 10.20.193.6:49068) 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be37fe640 1 -- 10.20.193.6:0/696820664 learned_addr learned my addr 10.20.193.6:0/696820664 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 1 --2- 10.20.193.6:0/696820664 >> v2:10.20.193.6:3300/0 conn(0x7f8be40c8b00 0x7f8be40c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 1 --2- 10.20.193.6:0/696820664 >> v2:10.20.193.40:3300/0 conn(0x7f8be41539a0 0x7f8be40c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 10 monclient(hunting): get_auth_request con 0x7f8be40c8b00 auth_method 0 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8be41539a0 auth_method 0 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:42.651 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be8ba8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.649+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be2ffd640 1 -- 10.20.193.6:0/696820664 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3673202692 0 0) 0x7f8bd4002920 con 0x7f8be4153d80 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be2ffd640 10 monclient(hunting): _negotiate 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be2ffd640 10 monclient(hunting): my global_id is 4322 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be2ffd640 1 -- 10.20.193.6:0/696820664 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bc8003620 con 0x7f8be4153d80 2026-02-22T23:22:42.652 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 10 monclient(hunting): handle_auth_done global_id 4285 payload 274 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 1 -- 10.20.193.6:0/696820664 >> v2:10.20.193.6:3300/0 conn(0x7f8be40c8b00 msgr2=0x7f8be40c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 1 --2- 10.20.193.6:0/696820664 >> v2:10.20.193.6:3300/0 conn(0x7f8be40c8b00 0x7f8be40c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 1 -- 10.20.193.6:0/696820664 >> v1:10.20.193.80:6789/0 conn(0x7f8be4153d80 legacy=0x7f8be4074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 10 monclient: _finish_hunting 0 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 1 monclient: found mon.noname-b 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.650+0000 7f8be3fff640 1 -- 10.20.193.6:0/696820664 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be40c5710 con 0x7f8be41539a0 2026-02-22T23:22:42.653 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.651+0000 7f8be3fff640 1 --2- 10.20.193.6:0/696820664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be41539a0 0x7f8be40c7270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8be4069b30 tx=0x7f8bd002ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9603cab54e0176a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 1 -- 10.20.193.6:0/696820664 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd003c070 con 0x7f8be41539a0 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: dump: 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:42.655 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: _finish_auth 0 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 5 monclient: authenticate success, global_id 4285 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 1 -- 10.20.193.6:0/696820664 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd002fa60 con 0x7f8be41539a0 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 1 -- 10.20.193.6:0/696820664 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd002fe10 con 0x7f8be41539a0 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8be2ffd640 10 monclient: dump: 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:42.656 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 4 set_mon_vals no callback set 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 10 monclient: shutdown 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 1 -- 10.20.193.6:0/696820664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be41539a0 msgr2=0x7f8be40c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 1 --2- 10.20.193.6:0/696820664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be41539a0 0x7f8be40c7270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8be4069b30 tx=0x7f8bd002ed50 comp rx=0 tx=0).stop 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 1 -- 10.20.193.6:0/696820664 shutdown_connections 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 1 --2- 10.20.193.6:0/696820664 >> v2:10.20.193.6:3300/0 conn(0x7f8be40c8b00 0x7f8be40c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 1 --2- 10.20.193.6:0/696820664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be41539a0 0x7f8be40c7270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.653+0000 7f8bea632640 1 -- 10.20.193.6:0/696820664 >> 10.20.193.6:0/696820664 conn(0x7f8be40c9f40 msgr2=0x7f8be40ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 -- 10.20.193.6:0/696820664 shutdown_connections 2026-02-22T23:22:42.657 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 -- 10.20.193.6:0/696820664 wait complete. 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 10 monclient: build_initial_monmap 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 10 monclient: monmap: 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 librados: starting msgr at 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 librados: starting objecter 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 Processor -- start 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 -- start start 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 librados: setting wanted keys 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 1 librados: calling monclient init 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.654+0000 7f8bea632640 10 monclient: init 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be40c8b00 0x7f8be41f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:42.658 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): picked mon.noname-b con 0x7f8be40c8b00 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8be41539a0 0x7f8be4074100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): picked mon.noname-a con 0x7f8be41539a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8be4153d80 0x7f8be40cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): picked mon.noname-c con 0x7f8be4153d80 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8be40c8300 con 0x7f8be41539a0 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8be40c8020 con 0x7f8be40c8b00 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8be40c85e0 con 0x7f8be4153d80 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8bea632640 10 monclient(hunting): authenticate will time out at 1037.383205s 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be37fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8be41539a0 0x7f8be4074100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be8ba8640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8be4153d80 0x7f8be40cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be37fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8be41539a0 0x7f8be4074100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:52096/0 (socket says 10.20.193.6:52096) 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be37fe640 1 -- 10.20.193.6:0/1670363830 learned_addr learned my addr 10.20.193.6:0/1670363830 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be37fe640 10 monclient(hunting): get_auth_request con 0x7f8be41539a0 auth_method 0 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be3fff640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be40c8b00 0x7f8be41f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be8ba8640 10 monclient(hunting): get_auth_request con 0x7f8be4153d80 auth_method 0 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be8ba8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be8ba8640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be8ba8640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be3fff640 10 monclient(hunting): get_auth_request con 0x7f8be40c8b00 auth_method 0 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.655+0000 7f8be3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be8ba8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be8ba8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be8ba8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:42.659 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.656+0000 7f8be3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 10 monclient(hunting): handle_auth_done global_id 4325 payload 824 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 1 -- 10.20.193.6:0/1670363830 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be40c8b00 msgr2=0x7f8be41f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be40c8b00 0x7f8be41f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 1 -- 10.20.193.6:0/1670363830 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8be41539a0 msgr2=0x7f8be4074100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8be41539a0 0x7f8be4074100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 10 monclient: _finish_hunting 0 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 1 monclient: found mon.noname-c 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.657+0000 7f8be8ba8640 1 -- 10.20.193.6:0/1670363830 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be41f47e0 con 0x7f8be4153d80 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.658+0000 7f8be8ba8640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8be4153d80 0x7f8be40cb980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800efc0 tx=0x7f8bd800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.658+0000 7f8be0ff9640 10 client.?.objecter ms_handle_connect 0x7f8be4153d80 2026-02-22T23:22:42.660 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.658+0000 7f8be0ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 1 -- 10.20.193.6:0/1670363830 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8018020 con 0x7f8be4153d80 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: dump: 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: _finish_auth 0 2026-02-22T23:22:42.662 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 5 monclient: authenticate success, global_id 4325 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 10 monclient: _renew_subs 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be41f3c60 con 0x7f8be4153d80 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 10 client.4325.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 10 monclient: _renew_subs 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be41f4220 con 0x7f8be4153d80 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8bea632640 1 librados: init done 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 1 -- 10.20.193.6:0/1670363830 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8014480 con 0x7f8be4153d80 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be27fc640 4 set_mon_vals no callback set 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 1 -- 10.20.193.6:0/1670363830 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8014830 con 0x7f8be4153d80 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.660+0000 7f8be0ff9640 10 monclient: dump: 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:42.663 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:42.664 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:42.664 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.661+0000 7f8bea632640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:42.664 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.661+0000 7f8bea632640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:42.664 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.661+0000 7f8bea632640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:42.664 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.661+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be40c52d0 con 0x7f8be4153d80 2026-02-22T23:22:42.674 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.672+0000 7f8be0ff9640 1 -- 10.20.193.6:0/1670363830 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f8bd80149d0 con 0x7f8be4153d80 2026-02-22T23:22:42.674 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.672+0000 7f8be0ff9640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8bb4078d70 0x7f8bb407b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:42.675 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.673+0000 7f8be3fff640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8bb4078d70 0x7f8bb407b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:42.675 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.673+0000 7f8be3fff640 10 monclient: get_auth_request con 0x7f8bb4078d70 auth_method 0 2026-02-22T23:22:42.675 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.673+0000 7f8be0ff9640 10 client.4325.objecter ms_handle_connect 0x7f8bb4078d70 2026-02-22T23:22:42.675 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.673+0000 7f8be3fff640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8bb4078d70 0x7f8bb407b230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0039000 tx=0x7f8bd0033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:42.676 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.674+0000 7f8be0ff9640 1 -- 10.20.193.6:0/1670363830 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f8bd80986d0 con 0x7f8be4153d80 2026-02-22T23:22:42.676 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.674+0000 7f8be0ff9640 10 client.4325.objecter ms_dispatch2 0x7f8be4000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:42.676 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.674+0000 7f8be0ff9640 3 client.4325.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:42.676 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.674+0000 7f8be0ff9640 3 client.4325.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:42.676 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.674+0000 7f8be0ff9640 20 client.4325.objecter dump_active .. 0 homeless 2026-02-22T23:22:42.782 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.780+0000 7f8be0ff9640 1 -- 10.20.193.6:0/1670363830 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8bd8022030 con 0x7f8be4153d80 2026-02-22T23:22:42.782 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.780+0000 7f8be0ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:42.782 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.780+0000 7f8be0ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:42.872 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.870+0000 7f8bea632640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:42.872 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.870+0000 7f8bea632640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:42.873 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.870+0000 7f8bea632640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:42.873 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.870+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8be4070af0 con 0x7f8be4153d80 2026-02-22T23:22:42.883 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.880+0000 7f8be0ff9640 1 -- 10.20.193.6:0/1670363830 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f8bd8010040 con 0x7f8be4153d80 2026-02-22T23:22:42.883 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.880+0000 7f8be0ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:42.883 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.880+0000 7f8be0ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:42.884 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:42.884 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:42.886 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 10 librados: watch_flush enter 2026-02-22T23:22:42.886 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 10 librados: watch_flush exit 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 20 client.4325.objecter shutdown clearing up homeless session... 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 10 client.4325.objecter successfully canceled tick 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8bb4078d70 msgr2=0x7f8bb407b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8bb4078d70 0x7f8bb407b230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0039000 tx=0x7f8bd0033000 comp rx=0 tx=0).stop 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 10 monclient: shutdown 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8be4153d80 msgr2=0x7f8be40cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8be4153d80 0x7f8be40cb980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800efc0 tx=0x7f8bd800c490 comp rx=0 tx=0).stop 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 shutdown_connections 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8bb4078d70 0x7f8bb407b230 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8be4153d80 0x7f8be40cb980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.887 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8be41539a0 0x7f8be4074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.888 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 --2- 10.20.193.6:0/1670363830 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8be40c8b00 0x7f8be41f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:42.888 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 >> 10.20.193.6:0/1670363830 conn(0x7f8be40c9f40 msgr2=0x7f8be406f980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:42.888 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 shutdown_connections 2026-02-22T23:22:42.888 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 -- 10.20.193.6:0/1670363830 wait complete. 2026-02-22T23:22:42.888 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:42.883+0000 7f8bea632640 1 librados: shutdown 2026-02-22T23:22:42.894 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:22:50.504 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:50.568 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient: get_monmap_and_config 2026-02-22T23:22:50.568 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient: build_initial_monmap 2026-02-22T23:22:50.568 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient: monmap: 2026-02-22T23:22:50.568 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:50.569 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:50.569 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:50.564667+0000 2026-02-22T23:22:50.569 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:50.564667+0000 2026-02-22T23:22:50.569 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:50.569 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:50.569 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:50.569 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 1 Processor -- start 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 1 -- start start 2026-02-22T23:22:50.570 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient: init 2026-02-22T23:22:50.571 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:50.571 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient: _add_conns ranks=[3,1,4,0,2,5] 2026-02-22T23:22:50.571 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7feadc1539a0 0x7feadc0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:50.571 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient(hunting): picked mon.noname-a con 0x7feadc1539a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:22:50.571 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7feadc153d80 addr v1:10.20.193.40:6789/0 2026-02-22T23:22:50.571 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.563+0000 7feae0e24640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7feadc0c8b00 0x7feadc0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:50.571 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 10 monclient(hunting): picked mon.noname-b con 0x7feadc0c8b00 addr v2:10.20.193.40:3300/0 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feadc075070 con 0x7feadc153d80 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7feadc0c5cd0 con 0x7feadc1539a0 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7feadc0c9490 con 0x7feadc0c8b00 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feae0e24640 10 monclient(hunting): authenticate will time out at 1045.292028s 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feada575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7feadc1539a0 0x7feadc0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:50.572 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feadad76640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7feadc0c8b00 0x7feadc0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feada575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7feadc1539a0 0x7feadc0c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:33068/0 (socket says 10.20.193.6:33068) 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feada575640 1 -- 10.20.193.6:0/2993433298 learned_addr learned my addr 10.20.193.6:0/2993433298 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feada575640 10 monclient(hunting): get_auth_request con 0x7feadc1539a0 auth_method 0 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feada575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feada575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.564+0000 7feada575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feadad76640 10 monclient(hunting): get_auth_request con 0x7feadc0c8b00 auth_method 0 2026-02-22T23:22:50.573 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feadad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feadad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feadad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feada575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feada575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feada575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feadad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feadad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:50.574 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.565+0000 7feadad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:50.585 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 10 monclient(hunting): handle_auth_done global_id 4290 payload 274 2026-02-22T23:22:50.585 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 1 -- 10.20.193.6:0/2993433298 >> v2:10.20.193.40:3300/0 conn(0x7feadc0c8b00 msgr2=0x7feadc0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:22:50.585 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 1 --2- 10.20.193.6:0/2993433298 >> v2:10.20.193.40:3300/0 conn(0x7feadc0c8b00 0x7feadc0c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.585 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 1 -- 10.20.193.6:0/2993433298 >> v1:10.20.193.40:6789/0 conn(0x7feadc153d80 legacy=0x7feadc074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:50.585 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 10 monclient: _finish_hunting 0 2026-02-22T23:22:50.585 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 1 monclient: found mon.noname-a 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.566+0000 7feada575640 1 -- 10.20.193.6:0/2993433298 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc0c5710 con 0x7feadc1539a0 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.567+0000 7feada575640 1 --2- 10.20.193.6:0/2993433298 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc1539a0 0x7feadc0c7270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feadc070f60 tx=0x7fead002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b434052e35b310fa server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 1 -- 10.20.193.6:0/2993433298 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead003c070 con 0x7feadc1539a0 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: dump: 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:50.586 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: _finish_auth 0 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: _check_auth_tickets 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 1 -- 10.20.193.6:0/2993433298 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead002fb30 con 0x7feadc1539a0 2026-02-22T23:22:50.587 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7feae0e24640 5 monclient: authenticate success, global_id 4290 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7feae0e24640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 1 -- 10.20.193.6:0/2993433298 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0037440 con 0x7feadc1539a0 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7feae0e24640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7feae0e24640 4 set_mon_vals no callback set 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7fead9573640 10 monclient: dump: 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.588 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:50.589 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:50.589 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:50.589 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:50.589 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:50.589 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:50.589 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.569+0000 7feae0e24640 10 monclient: shutdown 2026-02-22T23:22:50.589 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 -- 10.20.193.6:0/2993433298 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc1539a0 msgr2=0x7feadc0c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 --2- 10.20.193.6:0/2993433298 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc1539a0 0x7feadc0c7270 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feadc070f60 tx=0x7fead002ee60 comp rx=0 tx=0).stop 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 -- 10.20.193.6:0/2993433298 shutdown_connections 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 --2- 10.20.193.6:0/2993433298 >> v2:10.20.193.40:3300/0 conn(0x7feadc0c8b00 0x7feadc0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 --2- 10.20.193.6:0/2993433298 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc1539a0 0x7feadc0c7270 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 -- 10.20.193.6:0/2993433298 >> 10.20.193.6:0/2993433298 conn(0x7feadc0c9f40 msgr2=0x7feadc0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 -- 10.20.193.6:0/2993433298 shutdown_connections 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 -- 10.20.193.6:0/2993433298 wait complete. 2026-02-22T23:22:50.590 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 10 monclient: build_initial_monmap 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 10 monclient: monmap: 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:50.591 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 librados: starting msgr at 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 librados: starting objecter 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 Processor -- start 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 -- start start 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 librados: setting wanted keys 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 1 librados: calling monclient init 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.570+0000 7feae0e24640 10 monclient: init 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feadc0c8b00 0x7feadc13fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): picked mon.noname-b con 0x7feadc0c8b00 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feadc1539a0 0x7feadc06d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:50.592 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): picked mon.noname-c con 0x7feadc1539a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc153d80 0x7feadc06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): picked mon.noname-a con 0x7feadc153d80 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7feadc0c8300 con 0x7feadc153d80 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7feadc0c8020 con 0x7feadc0c8b00 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7feadc0c85e0 con 0x7feadc1539a0 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feae0e24640 10 monclient(hunting): authenticate will time out at 1045.299204s 2026-02-22T23:22:50.593 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc153d80 0x7feadc06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc153d80 0x7feadc06dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:33078/0 (socket says 10.20.193.6:33078) 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 1 -- 10.20.193.6:0/1131594255 learned_addr learned my addr 10.20.193.6:0/1131594255 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 10 monclient(hunting): get_auth_request con 0x7feadc153d80 auth_method 0 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feada575640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feadc0c8b00 0x7feadc13fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7fead9d74640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feadc1539a0 0x7feadc06d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.571+0000 7feadad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7fead9d74640 10 monclient(hunting): get_auth_request con 0x7feadc1539a0 auth_method 0 2026-02-22T23:22:50.594 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7fead9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7fead9d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7fead9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7feada575640 10 monclient(hunting): get_auth_request con 0x7feadc0c8b00 auth_method 0 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7feada575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7feada575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7feada575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7feada575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7feada575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7feada575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7fead9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7fead9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.572+0000 7fead9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:50.595 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 10 monclient(hunting): handle_auth_done global_id 4293 payload 824 2026-02-22T23:22:50.596 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 1 -- 10.20.193.6:0/1131594255 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feadc1539a0 msgr2=0x7feadc06d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:50.596 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feadc1539a0 0x7feadc06d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.596 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 1 -- 10.20.193.6:0/1131594255 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feadc0c8b00 msgr2=0x7feadc13fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:50.596 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feadc0c8b00 0x7feadc13fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.596 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 10 monclient: _finish_hunting 0 2026-02-22T23:22:50.596 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 1 monclient: found mon.noname-a 2026-02-22T23:22:50.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:50.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:22:50.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.573+0000 7feadad76640 1 -- 10.20.193.6:0/1131594255 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1f3750 con 0x7feadc153d80 2026-02-22T23:22:50.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.574+0000 7feadad76640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc153d80 0x7feadc06dd90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feacc00efc0 tx=0x7feacc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:50.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.574+0000 7feac2ffd640 10 client.?.objecter ms_handle_connect 0x7feadc153d80 2026-02-22T23:22:50.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.574+0000 7feac2ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:50.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feac2ffd640 1 -- 10.20.193.6:0/1131594255 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc018020 con 0x7feadc153d80 2026-02-22T23:22:50.598 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feac2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:50.598 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feac2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:22:50.598 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feac2ffd640 10 monclient: dump: 2026-02-22T23:22:50.598 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:50.598 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:50.598 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feac2ffd640 10 monclient: _finish_auth 0 2026-02-22T23:22:50.599 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feac2ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feac2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 5 monclient: authenticate success, global_id 4293 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 10 monclient: _renew_subs 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc1f3a30 con 0x7feadc153d80 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 10 client.4293.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 10 monclient: _renew_subs 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1f3ff0 con 0x7feadc153d80 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.575+0000 7feae0e24640 1 librados: init done 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feae0e24640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:50.600 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feae0e24640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feae0e24640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc153180 con 0x7feadc153d80 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feac2ffd640 1 -- 10.20.193.6:0/1131594255 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc014480 con 0x7feadc153d80 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feac2ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feac2ffd640 1 -- 10.20.193.6:0/1131594255 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc014830 con 0x7feadc153d80 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feac2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feac2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7feac2ffd640 10 monclient: dump: 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.601 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:50.602 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:50.602 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:50.602 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:50.602 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:50.602 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:50.602 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:50.602 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.576+0000 7fead8d72640 4 set_mon_vals no callback set 2026-02-22T23:22:50.690 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.687+0000 7feac2ffd640 1 -- 10.20.193.6:0/1131594255 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7feacc0149d0 con 0x7feadc153d80 2026-02-22T23:22:50.690 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.687+0000 7feac2ffd640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feaa8078d70 0x7feaa807b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:50.690 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.687+0000 7feada575640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feaa8078d70 0x7feaa807b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:50.690 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.687+0000 7feada575640 10 monclient: get_auth_request con 0x7feaa8078d70 auth_method 0 2026-02-22T23:22:50.691 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.688+0000 7feac2ffd640 10 client.4293.objecter ms_handle_connect 0x7feaa8078d70 2026-02-22T23:22:50.691 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.688+0000 7feada575640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feaa8078d70 0x7feaa807b230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feadc06f660 tx=0x7fead0039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:50.698 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.695+0000 7feac2ffd640 1 -- 10.20.193.6:0/1131594255 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7feacc05f190 con 0x7feadc153d80 2026-02-22T23:22:50.698 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.695+0000 7feac2ffd640 10 client.4293.objecter ms_dispatch2 0x7feadc000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:50.698 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.695+0000 7feac2ffd640 3 client.4293.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:50.699 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.695+0000 7feac2ffd640 3 client.4293.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:50.699 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.696+0000 7feac2ffd640 20 client.4293.objecter dump_active .. 0 homeless 2026-02-22T23:22:50.699 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.696+0000 7feac2ffd640 1 -- 10.20.193.6:0/1131594255 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7feacc022030 con 0x7feadc153d80 2026-02-22T23:22:50.699 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.696+0000 7feac2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:50.699 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.696+0000 7feac2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:50.788 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.786+0000 7feae0e24640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:50.789 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.786+0000 7feae0e24640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:50.789 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.786+0000 7feae0e24640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:50.789 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.786+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feadc15e1e0 con 0x7feadc153d80 2026-02-22T23:22:50.799 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.796+0000 7feac2ffd640 1 -- 10.20.193.6:0/1131594255 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7feadc15e1e0 con 0x7feadc153d80 2026-02-22T23:22:50.799 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.796+0000 7feac2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:50.799 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.796+0000 7feac2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:50.799 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:50.800 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:50.802 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 10 librados: watch_flush enter 2026-02-22T23:22:50.802 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 10 librados: watch_flush exit 2026-02-22T23:22:50.802 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 20 client.4293.objecter shutdown clearing up homeless session... 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 10 client.4293.objecter successfully canceled tick 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feaa8078d70 msgr2=0x7feaa807b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feaa8078d70 0x7feaa807b230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feadc06f660 tx=0x7fead0039040 comp rx=0 tx=0).stop 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 10 monclient: shutdown 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc153d80 msgr2=0x7feadc06dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc153d80 0x7feadc06dd90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feacc00efc0 tx=0x7feacc00c490 comp rx=0 tx=0).stop 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 shutdown_connections 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feaa8078d70 0x7feaa807b230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feadc153d80 0x7feadc06dd90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feadc1539a0 0x7feadc06d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.803 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 --2- 10.20.193.6:0/1131594255 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feadc0c8b00 0x7feadc13fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:50.804 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 >> 10.20.193.6:0/1131594255 conn(0x7feadc0c9f40 msgr2=0x7feadc149350 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:50.804 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 shutdown_connections 2026-02-22T23:22:50.804 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 -- 10.20.193.6:0/1131594255 wait complete. 2026-02-22T23:22:50.804 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:50.799+0000 7feae0e24640 1 librados: shutdown 2026-02-22T23:22:50.812 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:22:52.760 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:37.010 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.763 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:37.013 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.781 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:37.031 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.791 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:37.002 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.792 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:37.003 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.795 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:37.041 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.796 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:37.043 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.810 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:37.022 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:52.815 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:37.061 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:22:58.423 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:22:58.489 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.486+0000 7f900ae05640 10 monclient: get_monmap_and_config 2026-02-22T23:22:58.489 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.486+0000 7f900ae05640 10 monclient: build_initial_monmap 2026-02-22T23:22:58.489 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.486+0000 7f900ae05640 10 monclient: monmap: 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:22:58.488306+0000 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:22:58.488306+0000 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:22:58.490 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:22:58.491 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:22:58.491 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:58.491 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 1 Processor -- start 2026-02-22T23:22:58.491 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 1 -- start start 2026-02-22T23:22:58.491 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient: init 2026-02-22T23:22:58.495 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:58.495 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient: _add_conns ranks=[2,1,0,3,5,4] 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f90041539a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f9004153d80 addr v1:10.20.193.40:6789/0 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f90040c8b00 addr v1:10.20.193.6:6789/0 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9004075070 con 0x7f90040c8b00 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90040c5710 con 0x7f9004153d80 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90040c59f0 con 0x7f90041539a0 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.487+0000 7f900ae05640 10 monclient(hunting): authenticate will time out at 1053.215667s 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.488+0000 7f8ffbfff640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f9004153d80 0x7f9004074960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:59850/0 (socket says 10.20.193.6:59850) 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.488+0000 7f8ffbfff640 1 -- 10.20.193.6:0/2884400684 learned_addr learned my addr 10.20.193.6:0/2884400684 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:22:58.496 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.488+0000 7f900937b640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f90040c8b00 0x7f90040c8f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:49882/0 (socket says 10.20.193.6:49882) 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 473900975 0 0) 0x7f9000002bd0 con 0x7f90040c8b00 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _negotiate 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): my global_id is 4296 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ff0003620 con 0x7f90040c8b00 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021296227 0 0) 0x7f90040c5710 con 0x7f9004153d80 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _negotiate 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): my global_id is 4300 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9000002bd0 con 0x7f9004153d80 2026-02-22T23:22:58.497 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1467558973 0 0) 0x7f90040c59f0 con 0x7f90041539a0 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _negotiate 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 10 monclient(hunting): my global_id is 4331 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.489+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90040c5710 con 0x7f90041539a0 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 764161108 0 0) 0x7f8ff0003620 con 0x7f90040c8b00 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 >> v1:10.20.193.80:6789/0 conn(0x7f90041539a0 legacy=0x7f90040c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 >> v1:10.20.193.40:6789/0 conn(0x7f9004153d80 legacy=0x7f9004074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:58.498 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90040c8020 con 0x7f90040c8b00 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: _finish_auth 0 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f9000003380 con 0x7f90040c8b00 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: dump: 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.499 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 >> v1:10.20.193.6:6789/0 conn(0x7f90040c8b00 legacy=0x7f90040c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f8ff0004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): picked mon.c con 0x7f8ff0004030 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:58.500 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff0004e40 0x7f8ff00052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): picked mon.b con 0x7f8ff0004e40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff0009ab0 0x7f8ff000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): picked mon.a con 0x7f8ff0009ab0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff000c610 con 0x7f8ff0009ab0 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff000c7c0 con 0x7f8ff0004e40 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff000c9c0 con 0x7f8ff0004030 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:58.501 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.491+0000 7f8ffb7fe640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f8ff0004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f8ffbfff640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff0004e40 0x7f8ff00052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 10 monclient(hunting): get_auth_request con 0x7f8ff0004030 auth_method 0 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f900937b640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff0009ab0 0x7f8ff000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f8ffbfff640 10 monclient(hunting): get_auth_request con 0x7f8ff0004e40 auth_method 0 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f8ffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f8ffbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f8ffbfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:22:58.502 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.492+0000 7f9008b7a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 10 monclient(hunting): get_auth_request con 0x7f8ff0009ab0 auth_method 0 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f8ffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f8ffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f8ffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:58.503 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 10 monclient(hunting): handle_auth_done global_id 4296 payload 293 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 1 -- 10.20.193.6:0/2884400684 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff0004e40 msgr2=0x7f8ff00052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff0004e40 0x7f8ff00052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 1 -- 10.20.193.6:0/2884400684 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff0009ab0 msgr2=0x7f8ff000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff0009ab0 0x7f8ff000bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 10 monclient: _finish_hunting 0 2026-02-22T23:22:58.510 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 1 monclient: found mon.c 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 1 -- 10.20.193.6:0/2884400684 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0003620 con 0x7f8ff0004030 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 10 monclient: _finish_auth 0 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 10 monclient: _check_auth_tickets 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f9008b7a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900937b640 1 -- 10.20.193.6:0/2884400684 reap_dead start 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900ae05640 5 monclient: authenticate success, global_id 4296 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.493+0000 7f900ae05640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.494+0000 7f9008b7a640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f8ff0004900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40031a0 tx=0x7f8ff402ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cd30b917422d4c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.495+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4002dc0 con 0x7f8ff0004030 2026-02-22T23:22:58.511 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.495+0000 7f8ffb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.495+0000 7f8ffb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.495+0000 7f8ffb7fe640 10 monclient: dump: 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:58.512 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.495+0000 7f900ae05640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff40052d0 con 0x7f8ff0004030 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f8ffb7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f8ffb7fe640 1 -- 10.20.193.6:0/2884400684 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff402f720 con 0x7f8ff0004030 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f8ffb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f8ffb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f8ffb7fe640 10 monclient: dump: 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.513 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 10 monclient: get_monmap_and_config success 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 4 set_mon_vals no callback set 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 10 monclient: shutdown 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 1 -- 10.20.193.6:0/2884400684 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 msgr2=0x7f8ff0004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f8ff0004900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40031a0 tx=0x7f8ff402ef00 comp rx=0 tx=0).stop 2026-02-22T23:22:58.514 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 1 -- 10.20.193.6:0/2884400684 shutdown_connections 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 1 --2- 10.20.193.6:0/2884400684 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f8ff0004900 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 1 -- 10.20.193.6:0/2884400684 >> 10.20.193.6:0/2884400684 conn(0x7f90040c9f40 msgr2=0x7f90040ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 1 -- 10.20.193.6:0/2884400684 shutdown_connections 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.496+0000 7f900ae05640 1 -- 10.20.193.6:0/2884400684 wait complete. 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient: build_initial_monmap 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient: monmap: 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:58.515 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 librados: starting msgr at 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 librados: starting objecter 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 Processor -- start 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 -- start start 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 librados: setting wanted keys 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 librados: calling monclient init 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient: init 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient: _reopen_session rank -1 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:22:58.516 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f900406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): picked mon.noname-c con 0x7f8ff0004030 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9004070130 0x7f9004072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): picked mon.noname-a con 0x7f9004070130 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f90040730a0 0x7f900406e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): picked mon.noname-b con 0x7f90040730a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f90040c85e0 con 0x7f9004070130 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f90040c5cd0 con 0x7f90040730a0 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): start opening mon connection 2026-02-22T23:22:58.517 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f90040c52d0 con 0x7f8ff0004030 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): _renew_subs 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.497+0000 7f900ae05640 10 monclient(hunting): authenticate will time out at 1053.225846s 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9004070130 0x7f9004072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f900406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f90040730a0 0x7f900406e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9004070130 0x7f9004072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39520/0 (socket says 10.20.193.6:39520) 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 1 -- 10.20.193.6:0/27794308 learned_addr learned my addr 10.20.193.6:0/27794308 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 10 monclient(hunting): get_auth_request con 0x7f9004070130 auth_method 0 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:58.518 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 10 monclient(hunting): get_auth_request con 0x7f8ff0004030 auth_method 0 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 10 monclient(hunting): get_auth_request con 0x7f90040730a0 auth_method 0 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f8ffbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:58.519 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f9008b7a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.498+0000 7f900937b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.499+0000 7f8ffbfff640 10 monclient(hunting): handle_auth_done global_id 4299 payload 824 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.499+0000 7f8ffbfff640 1 -- 10.20.193.6:0/27794308 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 msgr2=0x7f900406fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.499+0000 7f8ffbfff640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f900406fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.499+0000 7f8ffbfff640 1 -- 10.20.193.6:0/27794308 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f90040730a0 msgr2=0x7f900406e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.499+0000 7f8ffbfff640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f90040730a0 0x7f900406e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.500+0000 7f8ffbfff640 10 monclient: _finish_hunting 0 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.500+0000 7f8ffbfff640 1 monclient: found mon.noname-a 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.500+0000 7f8ffbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.500+0000 7f8ffbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:22:58.520 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.500+0000 7f8ffbfff640 1 -- 10.20.193.6:0/27794308 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90041f38c0 con 0x7f9004070130 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.501+0000 7f8ffbfff640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9004070130 0x7f9004072b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0070e0 tx=0x7f8fec0075b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.501+0000 7f8ff97fa640 10 client.?.objecter ms_handle_connect 0x7f9004070130 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.501+0000 7f8ff97fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.502+0000 7f8ff97fa640 1 -- 10.20.193.6:0/27794308 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00f7b0 con 0x7f9004070130 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.502+0000 7f8ff97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.502+0000 7f8ff97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.502+0000 7f8ff97fa640 10 monclient: dump: 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.521 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 10 monclient: _finish_auth 0 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 10 monclient: _check_auth_tickets 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 1 -- 10.20.193.6:0/27794308 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec018070 con 0x7f9004070130 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 5 monclient: authenticate success, global_id 4299 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 monclient: _renew_subs 2026-02-22T23:22:58.522 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90041f3ba0 con 0x7f9004070130 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 1 -- 10.20.193.6:0/27794308 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00fb70 con 0x7f9004070130 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ffaffd640 4 set_mon_vals no callback set 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f8ff97fa640 10 monclient: dump: 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:22:58.523 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 client.4299.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 monclient: _renew_subs 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90041f4160 con 0x7f9004070130 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 1 librados: init done 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:58.524 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:58.525 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:58.525 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.503+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90040c8f50 con 0x7f9004070130 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.516+0000 7f8ff97fa640 1 -- 10.20.193.6:0/27794308 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f8fec00fd10 con 0x7f9004070130 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.517+0000 7f8ff97fa640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fd4078da0 0x7f8fd407b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.517+0000 7f9008b7a640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fd4078da0 0x7f8fd407b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.517+0000 7f9008b7a640 10 monclient: get_auth_request con 0x7f8fd4078da0 auth_method 0 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.518+0000 7f8ff97fa640 10 client.4299.objecter ms_handle_connect 0x7f8fd4078da0 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.518+0000 7f9008b7a640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fd4078da0 0x7f8fd407b260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4037000 tx=0x7f8ff4009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.520+0000 7f8ff97fa640 1 -- 10.20.193.6:0/27794308 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f8fec05e1e0 con 0x7f9004070130 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.520+0000 7f8ff97fa640 10 client.4299.objecter ms_dispatch2 0x7f9004000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:22:58.526 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.520+0000 7f8ff97fa640 3 client.4299.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:22:58.527 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.520+0000 7f8ff97fa640 3 client.4299.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:22:58.527 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.520+0000 7f8ff97fa640 20 client.4299.objecter dump_active .. 0 homeless 2026-02-22T23:22:58.625 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.622+0000 7f8ff97fa640 1 -- 10.20.193.6:0/27794308 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8fec0628a0 con 0x7f9004070130 2026-02-22T23:22:58.625 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.622+0000 7f8ff97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:22:58.625 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.622+0000 7f8ff97fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:22:58.713 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.711+0000 7f900ae05640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:58.713 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.711+0000 7f900ae05640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:58.714 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.711+0000 7f900ae05640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:22:58.714 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.711+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90041477f0 con 0x7f9004070130 2026-02-22T23:22:58.724 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.722+0000 7f8ff97fa640 1 -- 10.20.193.6:0/27794308 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f8fec0621a0 con 0x7f9004070130 2026-02-22T23:22:58.724 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.722+0000 7f8ff97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:22:58.724 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.722+0000 7f8ff97fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:22:58.725 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:22:58.725 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:22:58.728 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 10 librados: watch_flush enter 2026-02-22T23:22:58.728 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 10 librados: watch_flush exit 2026-02-22T23:22:58.728 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 20 client.4299.objecter shutdown clearing up homeless session... 2026-02-22T23:22:58.728 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 10 client.4299.objecter successfully canceled tick 2026-02-22T23:22:58.728 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fd4078da0 msgr2=0x7f8fd407b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.728 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fd4078da0 0x7f8fd407b260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4037000 tx=0x7f8ff4009030 comp rx=0 tx=0).stop 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 10 monclient: shutdown 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9004070130 msgr2=0x7f9004072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.725+0000 7f900ae05640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9004070130 0x7f9004072b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0070e0 tx=0x7f8fec0075b0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 shutdown_connections 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fd4078da0 0x7f8fd407b260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f90040730a0 0x7f900406e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9004070130 0x7f9004072b60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 --2- 10.20.193.6:0/27794308 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff0004030 0x7f900406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 >> 10.20.193.6:0/27794308 conn(0x7f90040c9f40 msgr2=0x7f9004070880 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 shutdown_connections 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 -- 10.20.193.6:0/27794308 wait complete. 2026-02-22T23:22:58.729 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:22:58.726+0000 7f900ae05640 1 librados: shutdown 2026-02-22T23:22:58.737 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:23:06.348 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:23:06.411 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.409+0000 7f38d1c0b640 10 monclient: get_monmap_and_config 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.409+0000 7f38d1c0b640 10 monclient: build_initial_monmap 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.409+0000 7f38d1c0b640 10 monclient: monmap: 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:23:06.410530+0000 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:23:06.410530+0000 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:06.412 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 1 Processor -- start 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 1 -- start start 2026-02-22T23:23:06.413 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient: init 2026-02-22T23:23:06.414 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient: _add_conns ranks=[2,1,3,5,0,4] 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f38cc155c00 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f38cc155fe0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f38cc0c7e40 0x7f38cc166680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient(hunting): picked mon.noname-a con 0x7f38cc0c7e40 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0c82c0 con 0x7f38cc155fe0 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38cc0c85a0 con 0x7f38cc155c00 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f38cc0c8880 con 0x7f38cc0c7e40 2026-02-22T23:23:06.415 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:06.416 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.410+0000 7f38d1c0b640 10 monclient(hunting): authenticate will time out at 1061.137902s 2026-02-22T23:23:06.416 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.411+0000 7f38cb7fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f38cc155c00 0x7f38cc15d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:39192/0 (socket says 10.20.193.6:39192) 2026-02-22T23:23:06.416 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.411+0000 7f38cb7fe640 1 -- 10.20.193.6:0/3013589876 learned_addr learned my addr 10.20.193.6:0/3013589876 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:06.440 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.439+0000 7f38cbfff640 1 --2- 10.20.193.6:0/3013589876 >> v2:10.20.193.6:3300/0 conn(0x7f38cc0c7e40 0x7f38cc166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:06.441 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.439+0000 7f38cbfff640 10 monclient(hunting): get_auth_request con 0x7f38cc0c7e40 auth_method 0 2026-02-22T23:23:06.441 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.439+0000 7f38cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:06.441 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.439+0000 7f38cbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:06.441 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.439+0000 7f38cbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:06.442 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:06.442 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:06.442 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:06.443 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38ca7fc640 1 -- 10.20.193.6:0/3013589876 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1348433793 0 0) 0x7f38cc0c82c0 con 0x7f38cc155fe0 2026-02-22T23:23:06.443 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38ca7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:23:06.443 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38ca7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:06.443 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38ca7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:06.443 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38ca7fc640 10 monclient(hunting): my global_id is 4306 2026-02-22T23:23:06.443 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.440+0000 7f38ca7fc640 1 -- 10.20.193.6:0/3013589876 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b0003620 con 0x7f38cc155fe0 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 10 monclient(hunting): handle_auth_done global_id 4302 payload 274 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 1 -- 10.20.193.6:0/3013589876 >> v1:10.20.193.80:6789/0 conn(0x7f38cc155c00 legacy=0x7f38cc15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 1 -- 10.20.193.6:0/3013589876 >> v1:10.20.193.40:6789/0 conn(0x7f38cc155fe0 legacy=0x7f38cc15f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 1 monclient: found mon.noname-a 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:06.444 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.441+0000 7f38cbfff640 1 -- 10.20.193.6:0/3013589876 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc0c8b60 con 0x7f38cc0c7e40 2026-02-22T23:23:06.445 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.443+0000 7f38cbfff640 1 --2- 10.20.193.6:0/3013589876 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc0c7e40 0x7f38cc166680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009d90 tx=0x7f38c002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53874c7a61402730 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:06.450 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 1 -- 10.20.193.6:0/3013589876 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c003c070 con 0x7f38cc0c7e40 2026-02-22T23:23:06.450 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:06.450 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:06.450 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: dump: 2026-02-22T23:23:06.450 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:06.450 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:06.451 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 1 -- 10.20.193.6:0/3013589876 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c0037440 con 0x7f38cc0c7e40 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 5 monclient: authenticate success, global_id 4302 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 1 -- 10.20.193.6:0/3013589876 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0037810 con 0x7f38cc0c7e40 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38ca7fc640 10 monclient: dump: 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.452 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 4 set_mon_vals no callback set 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 10 monclient: shutdown 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 1 -- 10.20.193.6:0/3013589876 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc0c7e40 msgr2=0x7f38cc166680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:06.453 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/3013589876 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc0c7e40 0x7f38cc166680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009d90 tx=0x7f38c002f1e0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 1 -- 10.20.193.6:0/3013589876 shutdown_connections 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/3013589876 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc0c7e40 0x7f38cc166680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 1 -- 10.20.193.6:0/3013589876 >> 10.20.193.6:0/3013589876 conn(0x7f38cc0c9f40 msgr2=0x7f38cc0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 1 -- 10.20.193.6:0/3013589876 shutdown_connections 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.445+0000 7f38d1c0b640 1 -- 10.20.193.6:0/3013589876 wait complete. 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient: build_initial_monmap 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient: monmap: 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:06.454 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 librados: starting msgr at 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 librados: starting objecter 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 Processor -- start 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 -- start start 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 librados: setting wanted keys 2026-02-22T23:23:06.455 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 librados: calling monclient init 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient: init 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f38cc0c7e40 0x7f38cc073df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): picked mon.noname-c con 0x7f38cc0c7e40 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc155c00 0x7f38cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): picked mon.noname-a con 0x7f38cc155c00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f38cc155fe0 0x7f38cc1e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): picked mon.noname-b con 0x7f38cc155fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:06.456 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc0c9640 con 0x7f38cc155c00 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc15fa00 con 0x7f38cc155fe0 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc1f3960 con 0x7f38cc0c7e40 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.446+0000 7f38d1c0b640 10 monclient(hunting): authenticate will time out at 1061.173974s 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc155c00 0x7f38cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38cbfff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f38cc155fe0 0x7f38cc1e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc155c00 0x7f38cc06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39546/0 (socket says 10.20.193.6:39546) 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 1 -- 10.20.193.6:0/1947792767 learned_addr learned my addr 10.20.193.6:0/1947792767 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 10 monclient(hunting): get_auth_request con 0x7f38cc155c00 auth_method 0 2026-02-22T23:23:06.457 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38cbfff640 10 monclient(hunting): get_auth_request con 0x7f38cc155fe0 auth_method 0 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38cbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38cbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38cbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.447+0000 7f38caffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.448+0000 7f38cbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:06.458 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.448+0000 7f38cbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:06.459 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.448+0000 7f38cbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 10 monclient(hunting): handle_auth_done global_id 4305 payload 824 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 1 -- 10.20.193.6:0/1947792767 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f38cc0c7e40 msgr2=0x7f38cc073df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f38cc0c7e40 0x7f38cc073df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 1 -- 10.20.193.6:0/1947792767 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f38cc155fe0 msgr2=0x7f38cc1e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f38cc155fe0 0x7f38cc1e7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 1 monclient: found mon.noname-a 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:06.461 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.449+0000 7f38caffd640 1 -- 10.20.193.6:0/1947792767 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc1f3c40 con 0x7f38cc155c00 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.450+0000 7f38caffd640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc155c00 0x7f38cc06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00c500 tx=0x7f38bc00c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.450+0000 7f38a7fff640 10 client.?.objecter ms_handle_connect 0x7f38cc155c00 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.450+0000 7f38a7fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38a7fff640 1 -- 10.20.193.6:0/1947792767 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0052d0 con 0x7f38cc155c00 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38a7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38a7fff640 10 monclient: dump: 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:06.462 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38a7fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:06.463 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38a7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 5 monclient: authenticate success, global_id 4305 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 10 monclient: _renew_subs 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc1f3f20 con 0x7f38cc155c00 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 10 client.4305.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 10 monclient: _renew_subs 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc1f44e0 con 0x7f38cc155c00 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.451+0000 7f38d1c0b640 1 librados: init done 2026-02-22T23:23:06.464 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.452+0000 7f38d1c0b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:06.465 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.452+0000 7f38d1c0b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:06.465 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.452+0000 7f38d1c0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:06.465 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.452+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38cc15f2f0 con 0x7f38cc155c00 2026-02-22T23:23:06.563 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.561+0000 7f38a7fff640 1 -- 10.20.193.6:0/1947792767 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38bc005470 con 0x7f38cc155c00 2026-02-22T23:23:06.563 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.561+0000 7f38a7fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.561+0000 7f38a7fff640 1 -- 10.20.193.6:0/1947792767 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0038c0 con 0x7f38cc155c00 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.561+0000 7f38a7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.561+0000 7f38a7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.561+0000 7f38a7fff640 10 monclient: dump: 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:06.564 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:06.565 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:06.565 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:06.565 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:06.565 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:06.565 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:06.565 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.561+0000 7f38c9ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:06.566 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.562+0000 7f38a7fff640 1 -- 10.20.193.6:0/1947792767 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f38bc003b20 con 0x7f38cc155c00 2026-02-22T23:23:06.566 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.563+0000 7f38a7fff640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f389c078da0 0x7f389c07b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:06.566 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.563+0000 7f38cb7fe640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f389c078da0 0x7f389c07b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:06.566 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.563+0000 7f38cb7fe640 10 monclient: get_auth_request con 0x7f389c078da0 auth_method 0 2026-02-22T23:23:06.567 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.564+0000 7f38cb7fe640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f389c078da0 0x7f389c07b260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38b8007fd0 tx=0x7f38b8007b40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:06.567 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.564+0000 7f38a7fff640 10 client.4305.objecter ms_handle_connect 0x7f389c078da0 2026-02-22T23:23:06.576 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.574+0000 7f38a7fff640 1 -- 10.20.193.6:0/1947792767 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7f38bc05df60 con 0x7f38cc155c00 2026-02-22T23:23:06.577 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.574+0000 7f38a7fff640 10 client.4305.objecter ms_dispatch2 0x7f38cc000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:23:06.577 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.574+0000 7f38a7fff640 3 client.4305.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:23:06.577 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.574+0000 7f38a7fff640 3 client.4305.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:23:06.577 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.575+0000 7f38a7fff640 20 client.4305.objecter dump_active .. 0 homeless 2026-02-22T23:23:06.577 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.575+0000 7f38a7fff640 1 -- 10.20.193.6:0/1947792767 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f38bc062620 con 0x7f38cc155c00 2026-02-22T23:23:06.577 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.575+0000 7f38a7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:06.577 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.575+0000 7f38a7fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:06.667 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.666+0000 7f38d1c0b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:06.667 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.666+0000 7f38d1c0b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:06.668 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.666+0000 7f38d1c0b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:06.668 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.666+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38cc0c97c0 con 0x7f38cc155c00 2026-02-22T23:23:06.677 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.676+0000 7f38a7fff640 1 -- 10.20.193.6:0/1947792767 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7f38bc061f20 con 0x7f38cc155c00 2026-02-22T23:23:06.677 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.676+0000 7f38a7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:06.678 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.676+0000 7f38a7fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:06.678 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:23:06.678 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 10 librados: watch_flush enter 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 10 librados: watch_flush exit 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 20 client.4305.objecter shutdown clearing up homeless session... 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 10 client.4305.objecter successfully canceled tick 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f389c078da0 msgr2=0x7f389c07b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f389c078da0 0x7f389c07b260 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38b8007fd0 tx=0x7f38b8007b40 comp rx=0 tx=0).stop 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 10 monclient: shutdown 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc155c00 msgr2=0x7f38cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc155c00 0x7f38cc06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00c500 tx=0x7f38bc00c9d0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.680 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 shutdown_connections 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f389c078da0 0x7f389c07b260 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f38cc155fe0 0x7f38cc1e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f38cc155c00 0x7f38cc06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 --2- 10.20.193.6:0/1947792767 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f38cc0c7e40 0x7f38cc073df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 >> 10.20.193.6:0/1947792767 conn(0x7f38cc0c9f40 msgr2=0x7f38cc0cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 shutdown_connections 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 -- 10.20.193.6:0/1947792767 wait complete. 2026-02-22T23:23:06.681 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:06.679+0000 7f38d1c0b640 1 librados: shutdown 2026-02-22T23:23:06.689 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:23:12.792 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:57.042 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.795 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:57.045 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.803 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:01:57.053 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.824 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:57.035 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.825 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:57.036 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.825 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:57.071 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.826 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:57.072 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.831 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:01:57.043 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:12.833 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:01:57.079 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:14.300 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.362+0000 7fad1cb72640 10 monclient: get_monmap_and_config 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.362+0000 7fad1cb72640 10 monclient: build_initial_monmap 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.362+0000 7fad1cb72640 10 monclient: monmap: 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:23:14.362800+0000 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:23:14.362800+0000 2026-02-22T23:23:14.363 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:14.364 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.362+0000 7fad1cb72640 1 Processor -- start 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.362+0000 7fad1cb72640 1 -- start start 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.362+0000 7fad1cb72640 10 monclient: init 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient: _add_conns ranks=[3,1,5,4,2,0] 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fad18155c00 0x7fad1815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient(hunting): picked mon.noname-a con 0x7fad18155c00 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fad18155fe0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fad180c7e40 0x7fad18073310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient(hunting): picked mon.noname-c con 0x7fad180c7e40 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad180c82c0 con 0x7fad18155fe0 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fad180c85a0 con 0x7fad18155c00 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fad180c8880 con 0x7fad180c7e40 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad1cb72640 10 monclient(hunting): authenticate will time out at 1069.090257s 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad16d76640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fad180c7e40 0x7fad18073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad16575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fad18155c00 0x7fad1815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad15d74640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fad18155fe0 0x7fad1815f2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:50124/0 (socket says 10.20.193.6:50124) 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.363+0000 7fad15d74640 1 -- 10.20.193.6:0/515814317 learned_addr learned my addr 10.20.193.6:0/515814317 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16d76640 10 monclient(hunting): get_auth_request con 0x7fad180c7e40 auth_method 0 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:14.365 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16575640 10 monclient(hunting): get_auth_request con 0x7fad18155c00 auth_method 0 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.364+0000 7fad16575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 10 monclient(hunting): handle_auth_done global_id 4340 payload 274 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 1 -- 10.20.193.6:0/515814317 >> v2:10.20.193.6:3300/0 conn(0x7fad18155c00 msgr2=0x7fad1815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:14.366 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 1 --2- 10.20.193.6:0/515814317 >> v2:10.20.193.6:3300/0 conn(0x7fad18155c00 0x7fad1815d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.367 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 1 -- 10.20.193.6:0/515814317 >> v1:10.20.193.40:6789/0 conn(0x7fad18155fe0 legacy=0x7fad1815f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:14.367 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 10 monclient: _finish_hunting 0 2026-02-22T23:23:14.367 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 1 monclient: found mon.noname-c 2026-02-22T23:23:14.367 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:14.367 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:14.367 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.365+0000 7fad16d76640 1 -- 10.20.193.6:0/515814317 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad180c8b60 con 0x7fad180c7e40 2026-02-22T23:23:14.367 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.366+0000 7fad16d76640 1 --2- 10.20.193.6:0/515814317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad180c7e40 0x7fad18073310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad180c4bd0 tx=0x7fad0802f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e5e090ae5f2c0e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 1 -- 10.20.193.6:0/515814317 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0803c070 con 0x7fad180c7e40 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: dump: 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: _finish_auth 0 2026-02-22T23:23:14.369 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 1 -- 10.20.193.6:0/515814317 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0802fe40 con 0x7fad180c7e40 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 5 monclient: authenticate success, global_id 4340 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 1 -- 10.20.193.6:0/515814317 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08037660 con 0x7fad180c7e40 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad15573640 10 monclient: dump: 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:14.370 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 4 set_mon_vals no callback set 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 10 monclient: shutdown 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 -- 10.20.193.6:0/515814317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad180c7e40 msgr2=0x7fad18073310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 --2- 10.20.193.6:0/515814317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad180c7e40 0x7fad18073310 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad180c4bd0 tx=0x7fad0802f130 comp rx=0 tx=0).stop 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 -- 10.20.193.6:0/515814317 shutdown_connections 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 --2- 10.20.193.6:0/515814317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad180c7e40 0x7fad18073310 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 --2- 10.20.193.6:0/515814317 >> v2:10.20.193.6:3300/0 conn(0x7fad18155c00 0x7fad1815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 -- 10.20.193.6:0/515814317 >> 10.20.193.6:0/515814317 conn(0x7fad180c9f40 msgr2=0x7fad180ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 -- 10.20.193.6:0/515814317 shutdown_connections 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.368+0000 7fad1cb72640 1 -- 10.20.193.6:0/515814317 wait complete. 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient: build_initial_monmap 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient: monmap: 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 librados: starting msgr at 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 librados: starting objecter 2026-02-22T23:23:14.372 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 Processor -- start 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 -- start start 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 librados: setting wanted keys 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 librados: calling monclient init 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient: init 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad180c7e40 0x7fad18149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): picked mon.noname-b con 0x7fad180c7e40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad18155c00 0x7fad181501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): picked mon.noname-c con 0x7fad18155c00 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad18155fe0 0x7fad18150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): picked mon.noname-a con 0x7fad18155fe0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad180c9640 con 0x7fad18155fe0 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad1815d730 con 0x7fad180c7e40 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad181f8020 con 0x7fad18155c00 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.369+0000 7fad1cb72640 10 monclient(hunting): authenticate will time out at 1069.096931s 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad18155c00 0x7fad181501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad180c7e40 0x7fad18149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad18155fe0 0x7fad18150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad18155c00 0x7fad181501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:47046/0 (socket says 10.20.193.6:47046) 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 1 -- 10.20.193.6:0/2475979233 learned_addr learned my addr 10.20.193.6:0/2475979233 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 10 monclient(hunting): get_auth_request con 0x7fad18155c00 auth_method 0 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 10 monclient(hunting): get_auth_request con 0x7fad180c7e40 auth_method 0 2026-02-22T23:23:14.373 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 10 monclient(hunting): get_auth_request con 0x7fad18155fe0 auth_method 0 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad15d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.370+0000 7fad16d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 10 monclient(hunting): handle_auth_done global_id 4343 payload 824 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 1 -- 10.20.193.6:0/2475979233 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad180c7e40 msgr2=0x7fad18149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:14.374 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad180c7e40 0x7fad18149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 1 -- 10.20.193.6:0/2475979233 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad18155fe0 msgr2=0x7fad18150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad18155fe0 0x7fad18150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 10 monclient: _finish_hunting 0 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 1 monclient: found mon.noname-c 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.372+0000 7fad15d74640 1 -- 10.20.193.6:0/2475979233 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad181f3d50 con 0x7fad18155c00 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.373+0000 7fad15d74640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad18155c00 0x7fad181501e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad0000c4d0 tx=0x7fad0000c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.373+0000 7facfeffd640 10 client.?.objecter ms_handle_connect 0x7fad18155c00 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.373+0000 7facfeffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 1 -- 10.20.193.6:0/2475979233 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00017070 con 0x7fad18155c00 2026-02-22T23:23:14.375 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: dump: 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 5 monclient: authenticate success, global_id 4343 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 monclient: _renew_subs 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad181f3ff0 con 0x7fad18155c00 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 client.4343.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 monclient: _renew_subs 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad181f45b0 con 0x7fad18155c00 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 1 librados: init done 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 1 -- 10.20.193.6:0/2475979233 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad00005de0 con 0x7fad18155c00 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad1815d1f0 con 0x7fad18155c00 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:14.376 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 1 -- 10.20.193.6:0/2475979233 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00021060 con 0x7fad18155c00 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7fad14d72640 4 set_mon_vals no callback set 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.374+0000 7facfeffd640 10 monclient: dump: 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:14.377 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:14.482 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.481+0000 7facfeffd640 1 -- 10.20.193.6:0/2475979233 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7fad00007270 con 0x7fad18155c00 2026-02-22T23:23:14.483 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.482+0000 7facfeffd640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7face4078da0 0x7face407b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:14.483 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.482+0000 7fad16575640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7face4078da0 0x7face407b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:14.483 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.482+0000 7fad16575640 10 monclient: get_auth_request con 0x7face4078da0 auth_method 0 2026-02-22T23:23:14.484 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.482+0000 7facfeffd640 10 client.4343.objecter ms_handle_connect 0x7face4078da0 2026-02-22T23:23:14.484 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.482+0000 7fad16575640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7face4078da0 0x7face407b260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009c40 tx=0x7fad0c009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:14.492 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.491+0000 7facfeffd640 1 -- 10.20.193.6:0/2475979233 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2565+0+0 (secure 0 0 0) 0x7fad0005da40 con 0x7fad18155c00 2026-02-22T23:23:14.492 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.491+0000 7facfeffd640 10 client.4343.objecter ms_dispatch2 0x7fad18000b90 osd_map(18..18 src has 1..18) 2026-02-22T23:23:14.492 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.491+0000 7facfeffd640 3 client.4343.objecter handle_osd_map got epochs [18,18] > 0 2026-02-22T23:23:14.492 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.491+0000 7facfeffd640 3 client.4343.objecter handle_osd_map decoding full epoch 18 2026-02-22T23:23:14.492 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.491+0000 7facfeffd640 20 client.4343.objecter dump_active .. 0 homeless 2026-02-22T23:23:14.493 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.492+0000 7facfeffd640 1 -- 10.20.193.6:0/2475979233 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fad00061a00 con 0x7fad18155c00 2026-02-22T23:23:14.493 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.492+0000 7facfeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:14.493 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.492+0000 7facfeffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:14.583 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.581+0000 7fad1cb72640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:14.583 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.581+0000 7fad1cb72640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:14.583 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.581+0000 7fad1cb72640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:14.583 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.581+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad180c97c0 con 0x7fad18155c00 2026-02-22T23:23:14.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.596+0000 7facfeffd640 1 -- 10.20.193.6:0/2475979233 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12879 (secure 0 0 0) 0x7fad00065d00 con 0x7fad18155c00 2026-02-22T23:23:14.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.596+0000 7facfeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:14.597 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.596+0000 7facfeffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:14.598 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:23:14.598 INFO:teuthology.misc.health.trial006.stdout:{"epoch":18,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:21:45.101496+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 10 librados: watch_flush enter 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 10 librados: watch_flush exit 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 20 client.4343.objecter shutdown clearing up homeless session... 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 10 client.4343.objecter successfully canceled tick 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7face4078da0 msgr2=0x7face407b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7face4078da0 0x7face407b260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009c40 tx=0x7fad0c009290 comp rx=0 tx=0).stop 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 10 monclient: shutdown 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad18155c00 msgr2=0x7fad181501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad18155c00 0x7fad181501e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad0000c4d0 tx=0x7fad0000c9a0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 shutdown_connections 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7face4078da0 0x7face407b260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad18155fe0 0x7fad18150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad18155c00 0x7fad181501e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 --2- 10.20.193.6:0/2475979233 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad180c7e40 0x7fad18149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 >> 10.20.193.6:0/2475979233 conn(0x7fad180c9f40 msgr2=0x7fad18070810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 shutdown_connections 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 -- 10.20.193.6:0/2475979233 wait complete. 2026-02-22T23:23:14.601 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:14.599+0000 7fad1cb72640 1 librados: shutdown 2026-02-22T23:23:14.613 DEBUG:teuthology.misc:0 of 16 OSDs are up 2026-02-22T23:23:16.248 INFO:tasks.ceph.osd.2.trial080.stderr:2026-02-22T23:23:16.243+0000 77b8940 -1 osd.2 0 log_to_monitors true 2026-02-22T23:23:16.249 INFO:tasks.ceph.osd.14.trial080.stderr:2026-02-22T23:23:16.243+0000 77b8940 -1 osd.14 0 log_to_monitors true 2026-02-22T23:23:16.250 INFO:tasks.ceph.osd.6.trial080.stderr:2026-02-22T23:23:16.247+0000 77b8940 -1 osd.6 0 log_to_monitors true 2026-02-22T23:23:16.582 INFO:tasks.ceph.osd.10.trial080.stderr:2026-02-22T23:23:16.578+0000 77b8940 -1 osd.10 0 log_to_monitors true 2026-02-22T23:23:16.732 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-22T23:23:16.727+0000 77b8940 -1 osd.7 0 log_to_monitors true 2026-02-22T23:23:16.735 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-22T23:23:16.732+0000 77b8940 -1 osd.11 0 log_to_monitors true 2026-02-22T23:23:17.084 INFO:tasks.ceph.osd.9.trial040.stderr:2026-02-22T23:23:17.078+0000 77b8940 -1 osd.9 0 log_to_monitors true 2026-02-22T23:23:17.438 INFO:tasks.ceph.osd.13.trial040.stderr:2026-02-22T23:23:17.432+0000 77b8940 -1 osd.13 0 log_to_monitors true 2026-02-22T23:23:17.468 INFO:tasks.ceph.osd.8.trial006.stderr:2026-02-22T23:23:17.462+0000 77b8940 -1 osd.8 0 log_to_monitors true 2026-02-22T23:23:17.529 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-22T23:23:17.527+0000 77b8940 -1 osd.3 0 log_to_monitors true 2026-02-22T23:23:17.531 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-22T23:23:17.529+0000 77b8940 -1 osd.15 0 log_to_monitors true 2026-02-22T23:23:17.547 INFO:tasks.ceph.osd.5.trial040.stderr:2026-02-22T23:23:17.540+0000 77b8940 -1 osd.5 0 log_to_monitors true 2026-02-22T23:23:17.655 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:23:17.628+0000 77b8940 -1 osd.4 0 log_to_monitors true 2026-02-22T23:23:18.059 INFO:tasks.ceph.osd.1.trial040.stderr:2026-02-22T23:23:18.053+0000 77b8940 -1 osd.1 0 log_to_monitors true 2026-02-22T23:23:18.164 INFO:tasks.ceph.osd.12.trial006.stderr:2026-02-22T23:23:18.161+0000 77b8940 -1 osd.12 0 log_to_monitors true 2026-02-22T23:23:18.370 INFO:tasks.ceph.osd.0.trial006.stderr:2026-02-22T23:23:18.366+0000 77b8940 -1 osd.0 0 log_to_monitors true 2026-02-22T23:23:18.906 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-22T23:23:18.905+0000 d2b0640 -1 osd.11 0 waiting for initial osdmap 2026-02-22T23:23:18.914 INFO:tasks.ceph.osd.14.trial080.stderr:2026-02-22T23:23:18.910+0000 d2b0640 -1 osd.14 0 waiting for initial osdmap 2026-02-22T23:23:18.920 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-22T23:23:18.919+0000 d2b0640 -1 osd.7 0 waiting for initial osdmap 2026-02-22T23:23:18.921 INFO:tasks.ceph.osd.6.trial080.stderr:2026-02-22T23:23:18.918+0000 d2b0640 -1 osd.6 0 waiting for initial osdmap 2026-02-22T23:23:18.942 INFO:tasks.ceph.osd.10.trial080.stderr:2026-02-22T23:23:18.939+0000 dab1640 -1 osd.10 0 waiting for initial osdmap 2026-02-22T23:23:18.965 INFO:tasks.ceph.osd.2.trial080.stderr:2026-02-22T23:23:18.962+0000 d2b0640 -1 osd.2 0 waiting for initial osdmap 2026-02-22T23:23:18.970 INFO:tasks.ceph.osd.9.trial040.stderr:2026-02-22T23:23:18.964+0000 d2b0640 -1 osd.9 0 waiting for initial osdmap 2026-02-22T23:23:19.119 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-22T23:23:19.113+0000 11c76640 -1 osd.11 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:19.138 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-22T23:23:19.131+0000 11c76640 -1 osd.7 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:19.144 INFO:tasks.ceph.osd.14.trial080.stderr:2026-02-22T23:23:19.136+0000 11c76640 -1 osd.14 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:19.146 INFO:tasks.ceph.osd.10.trial080.stderr:2026-02-22T23:23:19.138+0000 15c7e640 -1 osd.10 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:19.160 INFO:tasks.ceph.osd.6.trial080.stderr:2026-02-22T23:23:19.151+0000 15c7e640 -1 osd.6 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:19.163 INFO:tasks.ceph.osd.2.trial080.stderr:2026-02-22T23:23:19.155+0000 11c76640 -1 osd.2 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:19.172 INFO:tasks.ceph.osd.9.trial040.stderr:2026-02-22T23:23:19.161+0000 11c76640 -1 osd.9 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:19.906 INFO:tasks.ceph.osd.8.trial006.stderr:2026-02-22T23:23:19.903+0000 d2b0640 -1 osd.8 0 waiting for initial osdmap 2026-02-22T23:23:19.913 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-22T23:23:19.912+0000 dab1640 -1 osd.15 0 waiting for initial osdmap 2026-02-22T23:23:19.928 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-22T23:23:19.927+0000 d2b0640 -1 osd.3 0 waiting for initial osdmap 2026-02-22T23:23:19.940 INFO:tasks.ceph.osd.12.trial006.stderr:2026-02-22T23:23:19.937+0000 d2b0640 -1 osd.12 0 waiting for initial osdmap 2026-02-22T23:23:19.940 INFO:tasks.ceph.osd.13.trial040.stderr:2026-02-22T23:23:19.934+0000 dab1640 -1 osd.13 0 waiting for initial osdmap 2026-02-22T23:23:19.954 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:23:19.951+0000 dab1640 -1 osd.4 0 waiting for initial osdmap 2026-02-22T23:23:19.969 INFO:tasks.ceph.osd.5.trial040.stderr:2026-02-22T23:23:19.962+0000 dab1640 -1 osd.5 0 waiting for initial osdmap 2026-02-22T23:23:20.004 INFO:tasks.ceph.osd.1.trial040.stderr:2026-02-22T23:23:19.998+0000 dab1640 -1 osd.1 0 waiting for initial osdmap 2026-02-22T23:23:20.145 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-22T23:23:20.139+0000 11c76640 -1 osd.15 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:20.145 INFO:tasks.ceph.osd.8.trial006.stderr:2026-02-22T23:23:20.137+0000 11c76640 -1 osd.8 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:20.152 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-22T23:23:20.146+0000 15c7e640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:20.168 INFO:tasks.ceph.osd.12.trial006.stderr:2026-02-22T23:23:20.161+0000 11c76640 -1 osd.12 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:20.181 INFO:tasks.ceph.osd.5.trial040.stderr:2026-02-22T23:23:20.170+0000 15c7e640 -1 osd.5 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:20.188 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:23:20.181+0000 11c76640 -1 osd.4 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:20.189 INFO:tasks.ceph.osd.13.trial040.stderr:2026-02-22T23:23:20.179+0000 11c76640 -1 osd.13 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:20.238 INFO:tasks.ceph.osd.1.trial040.stderr:2026-02-22T23:23:20.227+0000 15c7e640 -1 osd.1 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:21.062 INFO:tasks.ceph.osd.0.trial006.stderr:2026-02-22T23:23:21.059+0000 d2b0640 -1 osd.0 0 waiting for initial osdmap 2026-02-22T23:23:21.342 INFO:tasks.ceph.osd.0.trial006.stderr:2026-02-22T23:23:21.333+0000 11c76640 -1 osd.0 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:23:22.222 DEBUG:teuthology.orchestra.run.trial006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:23:22.281 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.279+0000 7f6b3490b640 10 monclient: get_monmap_and_config 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.279+0000 7f6b3490b640 10 monclient: build_initial_monmap 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.279+0000 7f6b3490b640 10 monclient: monmap: 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:23:22.280608+0000 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:23:22.280608+0000 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:22.282 INFO:teuthology.misc.health.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 Processor -- start 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 -- start start 2026-02-22T23:23:22.283 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient: init 2026-02-22T23:23:22.291 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:22.291 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient: _add_conns ranks=[5,4,3,0,2,1] 2026-02-22T23:23:22.291 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6b2c1539a0 0x7f6b2c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): picked mon.noname-c con 0x7f6b2c1539a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f6b2c153d80 0x7f6b2c074960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): picked mon.noname-b con 0x7f6b2c153d80 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f6b2c0c8b00 0x7f6b2c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): picked mon.noname-a con 0x7f6b2c0c8b00 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6b2c0c5cd0 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f6b2c0c9490 con 0x7f6b2c153d80 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6b2c0c5710 con 0x7f6b2c1539a0 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:22.292 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.280+0000 7f6b3490b640 10 monclient(hunting): authenticate will time out at 1077.008009s 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32680640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6b2c1539a0 0x7f6b2c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32e81640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f6b2c0c8b00 0x7f6b2c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b31e7f640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f6b2c153d80 0x7f6b2c074960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32680640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6b2c1539a0 0x7f6b2c0c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43190/0 (socket says 10.20.193.6:43190) 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32680640 1 -- 10.20.193.6:0/2630363134 learned_addr learned my addr 10.20.193.6:0/2630363134 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32680640 10 monclient(hunting): get_auth_request con 0x7f6b2c1539a0 auth_method 0 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32680640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32680640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32e81640 10 monclient(hunting): get_auth_request con 0x7f6b2c0c8b00 auth_method 0 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32e81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32e81640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b32e81640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.293 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b31e7f640 10 monclient(hunting): get_auth_request con 0x7f6b2c153d80 auth_method 0 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b31e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b31e7f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.281+0000 7f6b31e7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32e81640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32e81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32e81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b31e7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b31e7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b31e7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 10 monclient(hunting): handle_auth_done global_id 4394 payload 274 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 1 -- 10.20.193.6:0/2630363134 >> v2:10.20.193.40:3300/0 conn(0x7f6b2c153d80 msgr2=0x7f6b2c074960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:22.294 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 1 --2- 10.20.193.6:0/2630363134 >> v2:10.20.193.40:3300/0 conn(0x7f6b2c153d80 0x7f6b2c074960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 1 -- 10.20.193.6:0/2630363134 >> v2:10.20.193.6:3300/0 conn(0x7f6b2c0c8b00 msgr2=0x7f6b2c0c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 1 --2- 10.20.193.6:0/2630363134 >> v2:10.20.193.6:3300/0 conn(0x7f6b2c0c8b00 0x7f6b2c0c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 10 monclient: _finish_hunting 0 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 1 monclient: found mon.noname-c 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.282+0000 7f6b32680640 1 -- 10.20.193.6:0/2630363134 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c0c59f0 con 0x7f6b2c1539a0 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.283+0000 7f6b32680640 1 --2- 10.20.193.6:0/2630363134 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c1539a0 0x7f6b2c0c7270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c06f660 tx=0x7f6b2402edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5541db72585664a6 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 1 -- 10.20.193.6:0/2630363134 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2403c070 con 0x7f6b2c1539a0 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: dump: 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.295 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: _finish_auth 0 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: _check_auth_tickets 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3490b640 5 monclient: authenticate success, global_id 4394 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3490b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 1 -- 10.20.193.6:0/2630363134 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2402fab0 con 0x7f6b2c1539a0 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 1 -- 10.20.193.6:0/2630363134 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24037440 con 0x7f6b2c1539a0 2026-02-22T23:23:22.296 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3167e640 10 monclient: dump: 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3490b640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.285+0000 7f6b3490b640 4 set_mon_vals no callback set 2026-02-22T23:23:22.297 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 10 monclient: shutdown 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 -- 10.20.193.6:0/2630363134 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c1539a0 msgr2=0x7f6b2c0c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 --2- 10.20.193.6:0/2630363134 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c1539a0 0x7f6b2c0c7270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c06f660 tx=0x7f6b2402edf0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 -- 10.20.193.6:0/2630363134 shutdown_connections 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 --2- 10.20.193.6:0/2630363134 >> v2:10.20.193.6:3300/0 conn(0x7f6b2c0c8b00 0x7f6b2c0c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 --2- 10.20.193.6:0/2630363134 >> v2:10.20.193.40:3300/0 conn(0x7f6b2c153d80 0x7f6b2c074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 --2- 10.20.193.6:0/2630363134 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c1539a0 0x7f6b2c0c7270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 -- 10.20.193.6:0/2630363134 >> 10.20.193.6:0/2630363134 conn(0x7f6b2c0c9f40 msgr2=0x7f6b2c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 -- 10.20.193.6:0/2630363134 shutdown_connections 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 -- 10.20.193.6:0/2630363134 wait complete. 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 10 monclient: build_initial_monmap 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 10 monclient: monmap: 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:epoch 0 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:created 0.000000 2026-02-22T23:23:22.298 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 librados: starting msgr at 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 librados: starting objecter 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 Processor -- start 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 -- start start 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 librados: setting wanted keys 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 1 librados: calling monclient init 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.286+0000 7f6b3490b640 10 monclient: init 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c0c8b00 0x7f6b2c06ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): picked mon.noname-c con 0x7f6b2c0c8b00 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:22.299 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b2c1539a0 0x7f6b2c073da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): picked mon.noname-b con 0x7f6b2c1539a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b2c153d80 0x7f6b2c0742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): picked mon.noname-a con 0x7f6b2c153d80 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c0c85e0 con 0x7f6b2c153d80 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c0c8300 con 0x7f6b2c1539a0 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c0c77b0 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b3490b640 10 monclient(hunting): authenticate will time out at 1077.014250s 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c0c8b00 0x7f6b2c06ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32e81640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b2c153d80 0x7f6b2c0742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b31e7f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b2c1539a0 0x7f6b2c073da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c0c8b00 0x7f6b2c06ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43194/0 (socket says 10.20.193.6:43194) 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 1 -- 10.20.193.6:0/3826560328 learned_addr learned my addr 10.20.193.6:0/3826560328 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:22.300 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 10 monclient(hunting): get_auth_request con 0x7f6b2c0c8b00 auth_method 0 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b31e7f640 10 monclient(hunting): get_auth_request con 0x7f6b2c1539a0 auth_method 0 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b31e7f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b31e7f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b31e7f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32e81640 10 monclient(hunting): get_auth_request con 0x7f6b2c153d80 auth_method 0 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32e81640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32e81640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32e81640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.287+0000 7f6b32680640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.288+0000 7f6b31e7f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.301 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.288+0000 7f6b31e7f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.288+0000 7f6b31e7f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.288+0000 7f6b32e81640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.288+0000 7f6b32e81640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.288+0000 7f6b32e81640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 10 monclient(hunting): handle_auth_done global_id 4397 payload 824 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 1 -- 10.20.193.6:0/3826560328 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b2c1539a0 msgr2=0x7f6b2c073da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b2c1539a0 0x7f6b2c073da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 1 -- 10.20.193.6:0/3826560328 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b2c153d80 msgr2=0x7f6b2c0742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b2c153d80 0x7f6b2c0742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 10 monclient: _finish_hunting 0 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 1 monclient: found mon.noname-c 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.289+0000 7f6b32680640 1 -- 10.20.193.6:0/3826560328 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c06d950 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.290+0000 7f6b32680640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c0c8b00 0x7f6b2c06ecd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b2402f350 tx=0x7f6b240047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.290+0000 7f6b16ffd640 10 client.?.objecter ms_handle_connect 0x7f6b2c0c8b00 2026-02-22T23:23:22.302 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.290+0000 7f6b16ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 1 -- 10.20.193.6:0/3826560328 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2403c040 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: dump: 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.304 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 5 monclient: authenticate success, global_id 4397 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 monclient: _renew_subs 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c06dc30 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 client.4397.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 monclient: _renew_subs 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c06e1f0 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 1 librados: init done 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:22.305 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c0c52d0 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 1 -- 10.20.193.6:0/3826560328 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2402f930 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 1 -- 10.20.193.6:0/3826560328 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24042830 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b16ffd640 10 monclient: dump: 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:epoch 1 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.306 INFO:teuthology.misc.health.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.307 INFO:teuthology.misc.health.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.307 INFO:teuthology.misc.health.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.307 INFO:teuthology.misc.health.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.307 INFO:teuthology.misc.health.trial006.stderr: 2026-02-22T23:23:22.307 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.291+0000 7f6b30e7d640 4 set_mon_vals no callback set 2026-02-22T23:23:22.396 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.395+0000 7f6b16ffd640 1 -- 10.20.193.6:0/3826560328 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f6b240429d0 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.396 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.395+0000 7f6b16ffd640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6afc078d70 0x7f6afc07b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.396 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.395+0000 7f6b31e7f640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6afc078d70 0x7f6afc07b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.396 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.395+0000 7f6b31e7f640 10 monclient: get_auth_request con 0x7f6afc078d70 auth_method 0 2026-02-22T23:23:22.396 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.396+0000 7f6b16ffd640 10 client.4397.objecter ms_handle_connect 0x7f6afc078d70 2026-02-22T23:23:22.397 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.396+0000 7f6b31e7f640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6afc078d70 0x7f6afc07b230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c072e10 tx=0x7f6b18009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:22.397 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.396+0000 7f6b16ffd640 1 -- 10.20.193.6:0/3826560328 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 8563+0+0 (secure 0 0 0) 0x7f6b24042cf0 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.398 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.396+0000 7f6b16ffd640 10 client.4397.objecter ms_dispatch2 0x7f6b2c000b90 osd_map(23..23 src has 1..23) 2026-02-22T23:23:22.398 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.396+0000 7f6b16ffd640 3 client.4397.objecter handle_osd_map got epochs [23,23] > 0 2026-02-22T23:23:22.398 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.396+0000 7f6b16ffd640 3 client.4397.objecter handle_osd_map decoding full epoch 23 2026-02-22T23:23:22.398 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.397+0000 7f6b16ffd640 20 client.4397.objecter dump_active .. 0 homeless 2026-02-22T23:23:22.404 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.403+0000 7f6b16ffd640 1 -- 10.20.193.6:0/3826560328 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6b24087670 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.404 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.403+0000 7f6b16ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:22.404 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.403+0000 7f6b16ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:22.489 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.488+0000 7f6b3490b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:22.490 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.488+0000 7f6b3490b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:22.490 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.488+0000 7f6b3490b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:22.490 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.488+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b2c1497b0 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.537 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.535+0000 7f6b16ffd640 1 -- 10.20.193.6:0/3826560328 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+22545 (secure 0 0 0) 0x7f6b2408b970 con 0x7f6b2c0c8b00 2026-02-22T23:23:22.537 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.535+0000 7f6b16ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:22.538 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.535+0000 7f6b16ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:22.538 INFO:teuthology.misc.health.trial006.stdout: 2026-02-22T23:23:22.538 INFO:teuthology.misc.health.trial006.stdout:{"epoch":23,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:23:21.874564+0000","last_up_change":"2026-02-22T23:23:21.874564+0000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:23:19.944772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6824","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6825","nonce":1913492598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6826","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6827","nonce":1913492598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6830","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6831","nonce":1913492598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6828","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6829","nonce":1913492598}]},"public_addr":"10.20.193.6:6825/1913492598","cluster_addr":"10.20.193.6:6827/1913492598","heartbeat_back_addr":"10.20.193.6:6831/1913492598","heartbeat_front_addr":"10.20.193.6:6829/1913492598","state":["exists","up"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6824","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6825","nonce":2813004313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6826","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6827","nonce":2813004313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6830","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6831","nonce":2813004313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6828","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6829","nonce":2813004313}]},"public_addr":"10.20.193.40:6825/2813004313","cluster_addr":"10.20.193.40:6827/2813004313","heartbeat_back_addr":"10.20.193.40:6831/2813004313","heartbeat_front_addr":"10.20.193.40:6829/2813004313","state":["exists","up"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6817","nonce":1223585399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6818","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6819","nonce":1223585399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6822","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6823","nonce":1223585399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6821","nonce":1223585399}]},"public_addr":"10.20.193.80:6817/1223585399","cluster_addr":"10.20.193.80:6819/1223585399","heartbeat_back_addr":"10.20.193.80:6823/1223585399","heartbeat_front_addr":"10.20.193.80:6821/1223585399","state":["exists","up"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6818","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6819","nonce":2996878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6820","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6821","nonce":2996878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6824","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6825","nonce":2996878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6822","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6823","nonce":2996878330}]},"public_addr":"10.20.193.189:6819/2996878330","cluster_addr":"10.20.193.189:6821/2996878330","heartbeat_back_addr":"10.20.193.189:6825/2996878330","heartbeat_front_addr":"10.20.193.189:6823/2996878330","state":["exists","up"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6800","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6801","nonce":2180113666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6802","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6803","nonce":2180113666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6806","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6807","nonce":2180113666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6804","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6805","nonce":2180113666}]},"public_addr":"10.20.193.6:6801/2180113666","cluster_addr":"10.20.193.6:6803/2180113666","heartbeat_back_addr":"10.20.193.6:6807/2180113666","heartbeat_front_addr":"10.20.193.6:6805/2180113666","state":["exists","up"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6800","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6801","nonce":2956321569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6802","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6803","nonce":2956321569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6806","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6807","nonce":2956321569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6804","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6805","nonce":2956321569}]},"public_addr":"10.20.193.40:6801/2956321569","cluster_addr":"10.20.193.40:6803/2956321569","heartbeat_back_addr":"10.20.193.40:6807/2956321569","heartbeat_front_addr":"10.20.193.40:6805/2956321569","state":["exists","up"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6801","nonce":144861518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6803","nonce":144861518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6807","nonce":144861518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6805","nonce":144861518}]},"public_addr":"10.20.193.80:6801/144861518","cluster_addr":"10.20.193.80:6803/144861518","heartbeat_back_addr":"10.20.193.80:6807/144861518","heartbeat_front_addr":"10.20.193.80:6805/144861518","state":["exists","up"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6810","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6811","nonce":980553521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6812","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6813","nonce":980553521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6816","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6817","nonce":980553521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6814","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6815","nonce":980553521}]},"public_addr":"10.20.193.189:6811/980553521","cluster_addr":"10.20.193.189:6813/980553521","heartbeat_back_addr":"10.20.193.189:6817/980553521","heartbeat_front_addr":"10.20.193.189:6815/980553521","state":["exists","up"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6816","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6817","nonce":1002652796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6818","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6819","nonce":1002652796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6822","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6823","nonce":1002652796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6820","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6821","nonce":1002652796}]},"public_addr":"10.20.193.6:6817/1002652796","cluster_addr":"10.20.193.6:6819/1002652796","heartbeat_back_addr":"10.20.193.6:6823/1002652796","heartbeat_front_addr":"10.20.193.6:6821/1002652796","state":["exists","up"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6808","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6809","nonce":979607499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6810","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6811","nonce":979607499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6814","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6815","nonce":979607499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6812","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6813","nonce":979607499}]},"public_addr":"10.20.193.40:6809/979607499","cluster_addr":"10.20.193.40:6811/979607499","heartbeat_back_addr":"10.20.193.40:6815/979607499","heartbeat_front_addr":"10.20.193.40:6813/979607499","state":["exists","up"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6809","nonce":2974128537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6810","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6811","nonce":2974128537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6814","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6815","nonce":2974128537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6813","nonce":2974128537}]},"public_addr":"10.20.193.80:6809/2974128537","cluster_addr":"10.20.193.80:6811/2974128537","heartbeat_back_addr":"10.20.193.80:6815/2974128537","heartbeat_front_addr":"10.20.193.80:6813/2974128537","state":["exists","up"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6802","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6803","nonce":2070530833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6804","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6805","nonce":2070530833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6808","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6809","nonce":2070530833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6806","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6807","nonce":2070530833}]},"public_addr":"10.20.193.189:6803/2070530833","cluster_addr":"10.20.193.189:6805/2070530833","heartbeat_back_addr":"10.20.193.189:6809/2070530833","heartbeat_front_addr":"10.20.193.189:6807/2070530833","state":["exists","up"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6808","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6809","nonce":138679770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6810","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6811","nonce":138679770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6814","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6815","nonce":138679770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6812","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6813","nonce":138679770}]},"public_addr":"10.20.193.6:6809/138679770","cluster_addr":"10.20.193.6:6811/138679770","heartbeat_back_addr":"10.20.193.6:6815/138679770","heartbeat_front_addr":"10.20.193.6:6813/138679770","state":["exists","up"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6816","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6817","nonce":1944897191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6818","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6819","nonce":1944897191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6822","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6823","nonce":1944897191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6820","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6821","nonce":1944897191}]},"public_addr":"10.20.193.40:6817/1944897191","cluster_addr":"10.20.193.40:6819/1944897191","heartbeat_back_addr":"10.20.193.40:6823/1944897191","heartbeat_front_addr":"10.20.193.40:6821/1944897191","state":["exists","up"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6824","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6825","nonce":3509265050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6826","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6827","nonce":3509265050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6830","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6831","nonce":3509265050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6828","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6829","nonce":3509265050}]},"public_addr":"10.20.193.80:6825/3509265050","cluster_addr":"10.20.193.80:6827/3509265050","heartbeat_back_addr":"10.20.193.80:6831/3509265050","heartbeat_front_addr":"10.20.193.80:6829/3509265050","state":["exists","up"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6826","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6827","nonce":3776650816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6828","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6829","nonce":3776650816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6832","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6833","nonce":3776650816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6830","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6831","nonce":3776650816}]},"public_addr":"10.20.193.189:6827/3776650816","cluster_addr":"10.20.193.189:6829/3776650816","heartbeat_back_addr":"10.20.193.189:6833/3776650816","heartbeat_front_addr":"10.20.193.189:6831/3776650816","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.477029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.595886+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.915624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.614574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.167676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.437161+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.930818+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.151987+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.520133+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.612032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:23:22.540 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 10 librados: watch_flush enter 2026-02-22T23:23:22.540 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 10 librados: watch_flush exit 2026-02-22T23:23:22.540 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 20 client.4397.objecter shutdown clearing up homeless session... 2026-02-22T23:23:22.540 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 10 client.4397.objecter successfully canceled tick 2026-02-22T23:23:22.540 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6afc078d70 msgr2=0x7f6afc07b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.540 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6afc078d70 0x7f6afc07b230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c072e10 tx=0x7f6b18009290 comp rx=0 tx=0).stop 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 10 monclient: shutdown 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c0c8b00 msgr2=0x7f6b2c06ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c0c8b00 0x7f6b2c06ecd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b2402f350 tx=0x7f6b240047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 shutdown_connections 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6afc078d70 0x7f6afc07b230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b2c153d80 0x7f6b2c0742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b2c1539a0 0x7f6b2c073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 --2- 10.20.193.6:0/3826560328 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b2c0c8b00 0x7f6b2c06ecd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 >> 10.20.193.6:0/3826560328 conn(0x7f6b2c0c9f40 msgr2=0x7f6b2c14f680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 shutdown_connections 2026-02-22T23:23:22.541 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 -- 10.20.193.6:0/3826560328 wait complete. 2026-02-22T23:23:22.542 INFO:teuthology.misc.health.trial006.stderr:2026-02-22T23:23:22.537+0000 7f6b3490b640 1 librados: shutdown 2026-02-22T23:23:22.546 DEBUG:teuthology.misc:16 of 16 OSDs are up 2026-02-22T23:23:22.546 INFO:tasks.ceph:Creating RBD pool 2026-02-22T23:23:22.546 DEBUG:teuthology.orchestra.run.trial006:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-22T23:23:22.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.609+0000 7f10c0a16640 10 monclient: get_monmap_and_config 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.609+0000 7f10c0a16640 10 monclient: build_initial_monmap 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.609+0000 7f10c0a16640 10 monclient: monmap: 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:22.610545+0000 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:22.610545+0000 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:22.612 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 1 Processor -- start 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 1 -- start start 2026-02-22T23:23:22.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient: init 2026-02-22T23:23:22.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:22.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient: _add_conns ranks=[3,4,2,5,0,1] 2026-02-22T23:23:22.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f10bc06ff20 0x7f10bc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient(hunting): picked mon.noname-a con 0x7f10bc06ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:22.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f10bc0c99a0 0x7f10bc159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient(hunting): picked mon.noname-b con 0x7f10bc0c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f10bc15a410 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10bc15a860 con 0x7f10bc15a410 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f10bc159a40 con 0x7f10bc06ff20 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f10bc0c7e90 con 0x7f10bc0c99a0 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.610+0000 7f10c0a16640 10 monclient(hunting): authenticate will time out at 1077.337951s 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f10bc06ff20 0x7f10bc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10b9d74640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f10bc0c99a0 0x7f10bc159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10bad76640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f10bc15a410 0x7f10bc14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:48064/0 (socket says 10.20.193.6:48064) 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10bad76640 1 -- 10.20.193.6:0/2266854091 learned_addr learned my addr 10.20.193.6:0/2266854091 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 10 monclient(hunting): get_auth_request con 0x7f10bc06ff20 auth_method 0 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10b9d74640 10 monclient(hunting): get_auth_request con 0x7f10bc0c99a0 auth_method 0 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10b9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10b9d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10b9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.611+0000 7f10ba575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.612+0000 7f10b9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.612+0000 7f10b9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.612+0000 7f10b9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.612+0000 7f10ba575640 10 monclient(hunting): handle_auth_done global_id 4365 payload 274 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.612+0000 7f10ba575640 1 -- 10.20.193.6:0/2266854091 >> v2:10.20.193.40:3300/0 conn(0x7f10bc0c99a0 msgr2=0x7f10bc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.612+0000 7f10ba575640 1 --2- 10.20.193.6:0/2266854091 >> v2:10.20.193.40:3300/0 conn(0x7f10bc0c99a0 0x7f10bc159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.613+0000 7f10ba575640 1 -- 10.20.193.6:0/2266854091 >> v1:10.20.193.80:6789/0 conn(0x7f10bc15a410 legacy=0x7f10bc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.613+0000 7f10ba575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.613+0000 7f10ba575640 1 monclient: found mon.noname-a 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.613+0000 7f10ba575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.613+0000 7f10ba575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.613+0000 7f10ba575640 1 -- 10.20.193.6:0/2266854091 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc0c8170 con 0x7f10bc06ff20 2026-02-22T23:23:22.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.613+0000 7f10ba575640 1 --2- 10.20.193.6:0/2266854091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc06ff20 0x7f10bc070300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10b00098a0 tx=0x7f10b002ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c7779733de9374 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:22.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 1 -- 10.20.193.6:0/2266854091 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b003c070 con 0x7f10bc06ff20 2026-02-22T23:23:22.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:22.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 10 monclient: dump: 2026-02-22T23:23:22.626 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:22.626 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 10 monclient: _finish_auth 0 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:22.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10b9573640 1 -- 10.20.193.6:0/2266854091 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b002fae0 con 0x7f10bc06ff20 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10c0a16640 5 monclient: authenticate success, global_id 4365 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.615+0000 7f10c0a16640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10b9573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10b9573640 1 -- 10.20.193.6:0/2266854091 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0037440 con 0x7f10bc06ff20 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 4 set_mon_vals no callback set 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10b9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10b9573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10b9573640 10 monclient: dump: 2026-02-22T23:23:22.628 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 10 monclient: shutdown 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 -- 10.20.193.6:0/2266854091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc06ff20 msgr2=0x7f10bc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 --2- 10.20.193.6:0/2266854091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc06ff20 0x7f10bc070300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f10b00098a0 tx=0x7f10b002ee10 comp rx=0 tx=0).stop 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 -- 10.20.193.6:0/2266854091 shutdown_connections 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 --2- 10.20.193.6:0/2266854091 >> v2:10.20.193.40:3300/0 conn(0x7f10bc0c99a0 0x7f10bc159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 --2- 10.20.193.6:0/2266854091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc06ff20 0x7f10bc070300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 -- 10.20.193.6:0/2266854091 >> 10.20.193.6:0/2266854091 conn(0x7f10bc0cd780 msgr2=0x7f10bc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 -- 10.20.193.6:0/2266854091 shutdown_connections 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 1 -- 10.20.193.6:0/2266854091 wait complete. 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 10 monclient: build_initial_monmap 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.616+0000 7f10c0a16640 10 monclient: monmap: 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:22.630 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 librados: starting msgr at 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 librados: starting objecter 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 Processor -- start 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 -- start start 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 librados: setting wanted keys 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 librados: calling monclient init 2026-02-22T23:23:22.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient: init 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f10bc06ff20 0x7f10bc06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): picked mon.noname-b con 0x7f10bc06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc0c99a0 0x7f10bc1f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): picked mon.noname-a con 0x7f10bc0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f10bc15a410 0x7f10bc0bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): picked mon.noname-c con 0x7f10bc15a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0c8a10 con 0x7f10bc0c99a0 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0c8730 con 0x7f10bc06ff20 2026-02-22T23:23:22.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc150520 con 0x7f10bc15a410 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.617+0000 7f10c0a16640 10 monclient(hunting): authenticate will time out at 1077.344730s 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc0c99a0 0x7f10bc1f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f10bc06ff20 0x7f10bc06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f10bc15a410 0x7f10bc0bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc0c99a0 0x7f10bc1f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58580/0 (socket says 10.20.193.6:58580) 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 1 -- 10.20.193.6:0/3290735664 learned_addr learned my addr 10.20.193.6:0/3290735664 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 10 monclient(hunting): get_auth_request con 0x7f10bc0c99a0 auth_method 0 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 10 monclient(hunting): get_auth_request con 0x7f10bc06ff20 auth_method 0 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 10 monclient(hunting): get_auth_request con 0x7f10bc15a410 auth_method 0 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10b9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10ba575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.618+0000 7f10bad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 10 monclient(hunting): handle_auth_done global_id 4368 payload 824 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 1 -- 10.20.193.6:0/3290735664 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f10bc15a410 msgr2=0x7f10bc0bfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f10bc15a410 0x7f10bc0bfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 1 -- 10.20.193.6:0/3290735664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f10bc06ff20 msgr2=0x7f10bc06ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f10bc06ff20 0x7f10bc06ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 10 monclient: _finish_hunting 0 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 1 monclient: found mon.noname-a 2026-02-22T23:23:22.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.619+0000 7f10b9d74640 1 -- 10.20.193.6:0/3290735664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc0c26c0 con 0x7f10bc0c99a0 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.621+0000 7f10b9d74640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc0c99a0 0x7f10bc1f4ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10a400ec10 tx=0x7f10a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.621+0000 7f10a2ffd640 10 client.?.objecter ms_handle_connect 0x7f10bc0c99a0 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.621+0000 7f10a2ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 1 -- 10.20.193.6:0/3290735664 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4017070 con 0x7f10bc0c99a0 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: dump: 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.636 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 5 monclient: authenticate success, global_id 4368 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 1 -- 10.20.193.6:0/3290735664 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a400cbe0 con 0x7f10bc0c99a0 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:22.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 1 -- 10.20.193.6:0/3290735664 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4013470 con 0x7f10bc0c99a0 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10a2ffd640 10 monclient: dump: 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:22.638 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10b8d72640 4 set_mon_vals no callback set 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 10 monclient: _renew_subs 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc0c29a0 con 0x7f10bc0c99a0 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 10 client.4368.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 10 monclient: _renew_subs 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc0c2f60 con 0x7f10bc0c99a0 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.622+0000 7f10c0a16640 1 librados: init done 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.623+0000 7f10c0a16640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.623+0000 7f10c0a16640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:22.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.623+0000 7f10c0a16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:22.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.623+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10bc070300 con 0x7f10bc0c99a0 2026-02-22T23:23:22.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.635+0000 7f10a2ffd640 1 -- 10.20.193.6:0/3290735664 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f10a4013610 con 0x7f10bc0c99a0 2026-02-22T23:23:22.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.635+0000 7f10a2ffd640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1088078dc0 0x7f108807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:22.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.635+0000 7f10ba575640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1088078dc0 0x7f108807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:22.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.635+0000 7f10ba575640 10 monclient: get_auth_request con 0x7f1088078dc0 auth_method 0 2026-02-22T23:23:22.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.636+0000 7f10a2ffd640 10 client.4368.objecter ms_handle_connect 0x7f1088078dc0 2026-02-22T23:23:22.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.636+0000 7f10ba575640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1088078dc0 0x7f108807b280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10b0039000 tx=0x7f10b0033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 1 -- 10.20.193.6:0/3290735664 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 8563+0+0 (secure 0 0 0) 0x7f10a405e520 con 0x7f10bc0c99a0 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 10 client.4368.objecter ms_dispatch2 0x7f10bc000b90 osd_map(23..23 src has 1..23) 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 3 client.4368.objecter handle_osd_map got epochs [23,23] > 0 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 3 client.4368.objecter handle_osd_map decoding full epoch 23 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 20 client.4368.objecter dump_active .. 0 homeless 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 1 -- 10.20.193.6:0/3290735664 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f10a40624e0 con 0x7f10bc0c99a0 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:22.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.733+0000 7f10a2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:22.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.816+0000 7f10c0a16640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-22T23:23:22.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.816+0000 7f10c0a16640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-22T23:23:22.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.816+0000 7f10c0a16640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:22.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:22.816+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f10bc14f5c0 con 0x7f10bc0c99a0 2026-02-22T23:23:23.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.042+0000 7f10a2ffd640 1 -- 10.20.193.6:0/3290735664 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v24) ==== 109+0+0 (secure 0 0 0) 0x7f10a40667e0 con 0x7f10bc0c99a0 2026-02-22T23:23:23.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.042+0000 7f10a2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2026-02-22T23:23:23.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.042+0000 7f10a2ffd640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2026-02-22T23:23:23.044 INFO:teuthology.orchestra.run.trial006.stderr:pool 'rbd' created 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 10 librados: watch_flush enter 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 10 librados: watch_flush exit 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 20 client.4368.objecter shutdown clearing up homeless session... 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 10 client.4368.objecter successfully canceled tick 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1088078dc0 msgr2=0x7f108807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1088078dc0 0x7f108807b280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10b0039000 tx=0x7f10b0033000 comp rx=0 tx=0).stop 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 10 monclient: shutdown 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc0c99a0 msgr2=0x7f10bc1f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc0c99a0 0x7f10bc1f4ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f10a400ec10 tx=0x7f10a400c520 comp rx=0 tx=0).stop 2026-02-22T23:23:23.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 shutdown_connections 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1088078dc0 0x7f108807b280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f10bc15a410 0x7f10bc0bfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f10bc0c99a0 0x7f10bc1f4ba0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 --2- 10.20.193.6:0/3290735664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f10bc06ff20 0x7f10bc06ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 >> 10.20.193.6:0/3290735664 conn(0x7f10bc0cd780 msgr2=0x7f10bc157e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 shutdown_connections 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 -- 10.20.193.6:0/3290735664 wait complete. 2026-02-22T23:23:23.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.045+0000 7f10c0a16640 1 librados: shutdown 2026-02-22T23:23:23.054 DEBUG:teuthology.orchestra.run.trial006:> rbd --cluster ceph pool init rbd 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.082+0000 7f1c3a5a40c0 10 monclient: get_monmap_and_config 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.082+0000 7f1c3a5a40c0 10 monclient: build_initial_monmap 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.082+0000 7f1c3a5a40c0 10 monclient: monmap: 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:23.083476+0000 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:23.083476+0000 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.083+0000 7f1c3a5a40c0 1 Processor -- start 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.083+0000 7f1c3a5a40c0 1 -- start start 2026-02-22T23:23:23.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.083+0000 7f1c3a5a40c0 10 monclient: init 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient: _reopen_session rank -1 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient: _add_conns ranks=[2,3,1,4,0,5] 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient(hunting): picked mon.noname-c-legacy con 0x557087208e30 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 1 --2- >> v2:10.20.193.6:3300/0 conn(0x557087209210 0x557087216630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient(hunting): picked mon.noname-a con 0x557087209210 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient(hunting): picked mon.noname-b-legacy con 0x55708713c360 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x557086fa4d20 con 0x55708713c360 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x55708717f870 con 0x557087208e30 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x557086fd1720 con 0x557087209210 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient(hunting): _renew_subs 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3a5a40c0 10 monclient(hunting): authenticate will time out at 1077.811095s 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3906a640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x557087208e30 0x557087213a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:48078/0 (socket says 10.20.193.6:48078) 2026-02-22T23:23:23.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.084+0000 7f1c3906a640 1 -- 10.20.193.6:0/509803782 learned_addr learned my addr 10.20.193.6:0/509803782 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 475110630 0 0) 0x557086fa4d20 con 0x55708713c360 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): _negotiate 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): my global_id is 4378 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c20001e10 con 0x55708713c360 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318819425 0 0) 0x55708717f870 con 0x557087208e30 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): _negotiate 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 10 monclient(hunting): my global_id is 4406 2026-02-22T23:23:23.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.086+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x557086fa4d20 con 0x557087208e30 2026-02-22T23:23:23.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.087+0000 7f1c38869640 1 --2- 10.20.193.6:0/509803782 >> v2:10.20.193.6:3300/0 conn(0x557087209210 0x557087216630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.087+0000 7f1c38869640 10 monclient(hunting): get_auth_request con 0x557087209210 auth_method 0 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.087+0000 7f1c38869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.087+0000 7f1c38869640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.087+0000 7f1c38869640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3998804947 0 0) 0x7f1c20001e10 con 0x55708713c360 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 >> v2:10.20.193.6:3300/0 conn(0x557087209210 msgr2=0x557087216630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 --2- 10.20.193.6:0/509803782 >> v2:10.20.193.6:3300/0 conn(0x557087209210 0x557087216630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 >> v1:10.20.193.80:6789/0 conn(0x557087208e30 legacy=0x557087213a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:23:23.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55708717fb50 con 0x55708713c360 2026-02-22T23:23:23.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: _finish_auth 0 2026-02-22T23:23:23.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:23.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:23.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f1c2c001a70 con 0x55708713c360 2026-02-22T23:23:23.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:23.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: dump: 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:23.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 >> v1:10.20.193.40:6789/0 conn(0x55708713c360 legacy=0x55708713c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c20003690 0x7f1c20003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): picked mon.b con 0x7f1c20003690 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x7f1c200025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): picked mon.a con 0x7f1c20002130 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c200082a0 0x7f1c2000a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): picked mon.c con 0x7f1c200082a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:23.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x557086fd1720 con 0x7f1c20002130 2026-02-22T23:23:23.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:23.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2000ad10 con 0x7f1c20003690 2026-02-22T23:23:23.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:23.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2000af10 con 0x7f1c200082a0 2026-02-22T23:23:23.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:23.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c2bfff640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:23.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.088+0000 7f1c38869640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x7f1c200025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c38869640 10 monclient(hunting): get_auth_request con 0x7f1c20002130 auth_method 0 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c38869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c38869640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c38869640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3906a640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c20003690 0x7f1c20003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c200082a0 0x7f1c2000a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c38869640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:23.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c38869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c38869640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 10 monclient(hunting): get_auth_request con 0x7f1c200082a0 auth_method 0 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3906a640 10 monclient(hunting): get_auth_request con 0x7f1c20003690 auth_method 0 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3906a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3906a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3906a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.089+0000 7f1c3a2f3640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c3906a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:23.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c3906a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c3906a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 10 monclient(hunting): handle_auth_done global_id 4378 payload 293 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 1 -- 10.20.193.6:0/509803782 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c200082a0 msgr2=0x7f1c2000a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c200082a0 0x7f1c2000a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 1 -- 10.20.193.6:0/509803782 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c20003690 msgr2=0x7f1c20003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c20003690 0x7f1c20003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 10 monclient: _finish_hunting 0 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 1 monclient: found mon.a 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 1 -- 10.20.193.6:0/509803782 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c20001e10 con 0x7f1c20002130 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 10 monclient: _finish_auth 0 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 10 monclient: _check_auth_tickets 2026-02-22T23:23:23.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c38869640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c3a2f3640 1 -- 10.20.193.6:0/509803782 reap_dead start 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c3a5a40c0 5 monclient: authenticate success, global_id 4378 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.090+0000 7f1c3a5a40c0 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.093+0000 7f1c38869640 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x7f1c200025b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400a7d0 tx=0x7f1c240588d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d204a7071a9b8f server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24066070 con 0x7f1c20002130 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 10 monclient: dump: 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.106 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c3a5a40c0 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2406a610 con 0x7f1c20002130 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 1 -- 10.20.193.6:0/509803782 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2400a420 con 0x7f1c20002130 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c2bfff640 10 monclient: dump: 2026-02-22T23:23:23.107 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c3a5a40c0 10 monclient: get_monmap_and_config success 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.095+0000 7f1c3a5a40c0 4 set_mon_vals no callback set 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 10 monclient: shutdown 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/509803782 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 msgr2=0x7f1c200025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:23.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x7f1c200025b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400a7d0 tx=0x7f1c240588d0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/509803782 shutdown_connections 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/509803782 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x7f1c200025b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/509803782 >> 10.20.193.6:0/509803782 conn(0x557087184e50 msgr2=0x557087185260 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/509803782 shutdown_connections 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/509803782 wait complete. 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 10 monclient: build_initial_monmap 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 10 monclient: monmap: 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:23.109 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 librados: starting msgr at 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 librados: starting objecter 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 Processor -- start 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 -- start start 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 librados: setting wanted keys 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 1 librados: calling monclient init 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.096+0000 7f1c3a5a40c0 10 monclient: init 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient: _reopen_session rank -1 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x5570872231c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): picked mon.noname-a con 0x7f1c20002130 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x55708720fbb0 0x557087225c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): picked mon.noname-b con 0x55708720fbb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x557087187480 0x557087207530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): picked mon.noname-c con 0x557087187480 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x557087021db0 con 0x7f1c20002130 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x557087025e80 con 0x55708720fbb0 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x557086fc0e50 con 0x557087187480 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): _renew_subs 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a5a40c0 10 monclient(hunting): authenticate will time out at 1077.824294s 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c38869640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x55708720fbb0 0x557087225c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3906a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x5570872231c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3906a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x5570872231c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58592/0 (socket says 10.20.193.6:58592) 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 learned_addr learned my addr 10.20.193.6:0/3963142441 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3906a640 10 monclient(hunting): get_auth_request con 0x7f1c20002130 auth_method 0 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3906a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3906a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3906a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c38869640 10 monclient(hunting): get_auth_request con 0x55708720fbb0 auth_method 0 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c38869640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c38869640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c38869640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.097+0000 7f1c3a2f3640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x557087187480 0x557087207530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3906a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:23.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3906a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3906a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c38869640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c38869640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c38869640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3a2f3640 10 monclient(hunting): get_auth_request con 0x557087187480 auth_method 0 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3a2f3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3a2f3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3a2f3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3a2f3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3a2f3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.098+0000 7f1c3a2f3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 10 monclient(hunting): handle_auth_done global_id 4374 payload 824 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x557087187480 msgr2=0x557087207530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x557087187480 0x557087207530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x55708720fbb0 msgr2=0x557087225c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:23.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x55708720fbb0 0x557087225c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 10 monclient: _finish_hunting 0 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 1 monclient: found mon.noname-a 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.099+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55708720e5c0 con 0x7f1c20002130 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.100+0000 7f1c3906a640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x5570872231c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c610 tx=0x7f1c3000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.100+0000 7f1c297fa640 10 client.?.objecter ms_handle_connect 0x7f1c20002130 2026-02-22T23:23:23.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.100+0000 7f1c297fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:23.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000f650 con 0x7f1c20002130 2026-02-22T23:23:23.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:23.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: dump: 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:23.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3000f830 con 0x7f1c20002130 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000fc20 con 0x7f1c20002130 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c297fa640 10 monclient: dump: 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:23.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 5 monclient: authenticate success, global_id 4374 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 10 monclient: _renew_subs 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55708720e830 con 0x7f1c20002130 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c2b7fe640 4 set_mon_vals no callback set 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 10 client.4374.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 10 monclient: _renew_subs 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55708720edf0 con 0x7f1c20002130 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 1 librados: init done 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.102+0000 7f1c3a5a40c0 10 librados: wait_for_osdmap waiting 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.113+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 4) ==== 99845+0+0 (secure 0 0 0) 0x7f1c30009070 con 0x7f1c20002130 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.113+0000 7f1c297fa640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 0x7f1c180792e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.113+0000 7f1c38869640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 0x7f1c180792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.113+0000 7f1c38869640 10 monclient: get_auth_request con 0x7f1c18076e20 auth_method 0 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.114+0000 7f1c38869640 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 msgr2=0x7f1c180792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2026-02-22T23:23:23.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.114+0000 7f1c38869640 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 msgr2=0x7f1c180792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.114+0000 7f1c38869640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 0x7f1c180792e0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f1c2406a000 tx=0x7f1c24066000 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.114+0000 7f1c38869640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 0x7f1c180792e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 8905+0+0 (secure 0 0 0) 0x7f1c3000fdc0 con 0x7f1c20002130 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c297fa640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_map(24..24 src has 1..24) 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map got epochs [24,24] > 0 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map decoding full epoch 24 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c297fa640 20 client.4374.objecter dump_active .. 0 homeless 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c3a5a40c0 10 librados: wait_for_osdmap done waiting 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c3a5a40c0 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c3a5a40c0 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c3a5a40c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:23.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.115+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x55708720f3b0 con 0x7f1c20002130 2026-02-22T23:23:23.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.315+0000 7f1c38869640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 0x7f1c180792e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:23.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.315+0000 7f1c38869640 10 monclient: get_auth_request con 0x7f1c18076e20 auth_method 0 2026-02-22T23:23:23.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.315+0000 7f1c297fa640 10 client.4374.objecter ms_handle_connect 0x7f1c18076e20 2026-02-22T23:23:23.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:23.315+0000 7f1c38869640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 0x7f1c180792e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c24000c00 tx=0x7f1c24064000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:24.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.142+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v25) ==== 141+0+0 (secure 0 0 0) 0x7f1c30060d00 con 0x7f1c20002130 2026-02-22T23:23:24.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.142+0000 7f1c297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2026-02-22T23:23:24.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.142+0000 7f1c297fa640 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2026-02-22T23:23:24.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.142+0000 7f1c3a5a40c0 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:24.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.142+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c2406a610 con 0x7f1c20002130 2026-02-22T23:23:24.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.238+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1c3000c040 con 0x7f1c20002130 2026-02-22T23:23:24.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.273+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 8 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c3000f650 con 0x7f1c20002130 2026-02-22T23:23:24.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.273+0000 7f1c297fa640 10 monclient: handle_get_version_reply finishing 1 version 25 2026-02-22T23:23:24.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.273+0000 7f1c2b7fe640 10 client.4374.objecter _get_latest_version latest 25, waiting 2026-02-22T23:23:24.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.273+0000 7f1c2b7fe640 10 client.4374.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:24.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.273+0000 7f1c2b7fe640 10 monclient: _renew_subs 2026-02-22T23:23:24.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.273+0000 7f1c2b7fe640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:24.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.273+0000 7f1c2b7fe640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=25}) -- 0x7f1c10003c90 con 0x7f1c20002130 2026-02-22T23:23:24.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 9 ==== osd_map(25..25 src has 1..25) ==== 682+0+0 (secure 0 0 0) 0x7f1c300711b0 con 0x7f1c20002130 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c297fa640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_map(25..25 src has 1..25) 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map got epochs [25,25] > 24 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map decoding incremental epoch 25 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c297fa640 20 client.4374.objecter dump_active .. 0 homeless 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 10 librados: create oid=rbd_trash nspace= 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 10 client.4374.objecter _op_submit op 0x557087207a70 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 20 client.4374.objecter _calc_target epoch 25 base rbd_trash @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 20 client.4374.objecter _calc_target target rbd_trash @2 -> pgid 2.5a749323 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 10 client.4374.objecter _calc_target raw pgid 2.5a749323 -> actual 2.3 acting [4,15] primary 4 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x557087187900 0x5570872017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:24.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 20 client.4374.objecter _get_session s=0x557086fc1230 osd=4 3 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 10 client.4374.objecter _op_submit oid rbd_trash '@2' '@2' [create] tid 1 osd.4 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 20 client.4374.objecter get_session s=0x557086fc1230 osd=4 3 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 15 client.4374.objecter _session_op_assign 4 1 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 15 client.4374.objecter _send_op 1 to 2.3 on osd.4 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557087143d20 con 0x557087187900 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 20 client.4374.objecter put_session s=0x557086fc1230 osd=4 4 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.274+0000 7f1c3a5a40c0 5 client.4374.objecter 1 in flight 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.275+0000 7f1c3a2f3640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x557087187900 0x5570872017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:24.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.275+0000 7f1c3a2f3640 10 monclient: get_auth_request con 0x557087187900 auth_method 0 2026-02-22T23:23:24.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.277+0000 7f1c297fa640 10 client.4374.objecter ms_handle_connect 0x557087187900 2026-02-22T23:23:24.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.277+0000 7f1c3a2f3640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x557087187900 0x5570872017f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 1 -- 10.20.193.6:0/3963142441 <== osd.4 v2:10.20.193.6:6800/2180113666 1 ==== osd_op_reply(1 rbd_trash [create] v25'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7f1c2c006600 con 0x557087187900 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_op_reply(1 rbd_trash [create] v25'1 uv1 ondisk = 0) 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 10 client.4374.objecter in handle_osd_op_reply 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 7 client.4374.objecter handle_osd_op_reply 1 ondisk uv 1 in 2.3 attempt 0 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 10 client.4374.objecter op 0 rval 0 len 0 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 15 client.4374.objecter handle_osd_op_reply completed tid 1 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 15 client.4374.objecter _finish_op 1 2026-02-22T23:23:24.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 20 client.4374.objecter put_session s=0x557086fc1230 osd=4 4 2026-02-22T23:23:24.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 15 client.4374.objecter _session_op_remove 4 1 2026-02-22T23:23:24.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a2f3640 5 client.4374.objecter 0 in flight 2026-02-22T23:23:24.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.768+0000 7f1c3a5a40c0 10 librados: Objecter returned from create r=0 2026-02-22T23:23:24.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 10 client.4374.objecter _op_submit op 0x55708727ef80 2026-02-22T23:23:24.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 20 client.4374.objecter _calc_target epoch 25 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 20 client.4374.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 10 client.4374.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x55708727fb30 0x557087281fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 20 client.4374.objecter _get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 10 client.4374.objecter _op_submit oid rbd_info '@2' '@2' [call rbd.metadata_list in=33b] tid 2 osd.3 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 20 client.4374.objecter get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 15 client.4374.objecter _session_op_assign 3 2 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 15 client.4374.objecter _send_op 2 to 2.2 on osd.3 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557087282540 con 0x55708727fb30 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.770+0000 7f1c3a5a40c0 5 client.4374.objecter 1 in flight 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.771+0000 7f1c3906a640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x55708727fb30 0x557087281fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:24.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.771+0000 7f1c3906a640 10 monclient: get_auth_request con 0x55708727fb30 auth_method 0 2026-02-22T23:23:24.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.773+0000 7f1c3906a640 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x55708727fb30 0x557087281fc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:24.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:24.773+0000 7f1c297fa640 10 client.4374.objecter ms_handle_connect 0x55708727fb30 2026-02-22T23:23:25.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 <== osd.3 v2:10.20.193.189:6818/2996878330 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f1c30020070 con 0x55708727fb30 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 10 client.4374.objecter in handle_osd_op_reply 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 7 client.4374.objecter handle_osd_op_reply 2 ondisk uv 0 in 2.2 attempt 0 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 10 client.4374.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 10 client.4374.objecter op 0 rval 0 len 0 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 15 client.4374.objecter handle_osd_op_reply completed tid 2 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 15 client.4374.objecter _finish_op 2 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 15 client.4374.objecter _session_op_remove 3 2 2026-02-22T23:23:25.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3906a640 5 client.4374.objecter 0 in flight 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 10 client.4374.objecter _op_submit op 0x55708727e160 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 20 client.4374.objecter _calc_target epoch 25 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_read 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 20 client.4374.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 10 client.4374.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 20 client.4374.objecter _get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 10 client.4374.objecter _op_submit oid rbd_info '@2' '@2' [read 0~0] tid 3 osd.3 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 20 client.4374.objecter get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 15 client.4374.objecter _session_op_assign 3 3 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 15 client.4374.objecter _send_op 3 to 2.2 on osd.3 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55708727e610 con 0x55708727fb30 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:25.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.095+0000 7f1c3a5a40c0 5 client.4374.objecter 1 in flight 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 <== osd.3 v2:10.20.193.189:6818/2996878330 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f1c30020070 con 0x55708727fb30 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 10 client.4374.objecter in handle_osd_op_reply 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 7 client.4374.objecter handle_osd_op_reply 3 ondisk uv 0 in 2.2 attempt 0 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 10 client.4374.objecter handle_osd_op_reply copying resulting 0 into existing ceph::buffer of length 0 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 10 client.4374.objecter op 0 rval 0 len 0 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 15 client.4374.objecter handle_osd_op_reply completed tid 3 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 15 client.4374.objecter _finish_op 3 2026-02-22T23:23:25.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:25.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 15 client.4374.objecter _session_op_remove 3 3 2026-02-22T23:23:25.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c3906a640 5 client.4374.objecter 0 in flight 2026-02-22T23:23:25.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c2affd640 10 client.4374.objecter allocate_selfmanaged_snap; pool: 2 2026-02-22T23:23:25.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c2affd640 10 client.4374.objecter pool_op_submit 4 2026-02-22T23:23:25.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c2affd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:25.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.101+0000 7f1c2affd640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7f1c140030d0 con 0x7f1c20002130 2026-02-22T23:23:25.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 10 ==== pool_op_reply(tid 4 (0) Success v26) ==== 55+0+0 (secure 0 0 0) 0x7f1c30065000 con 0x7f1c20002130 2026-02-22T23:23:25.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 pool_op_reply(tid 4 (0) Success v26) 2026-02-22T23:23:25.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 10 client.4374.objecter handle_pool_op_reply pool_op_reply(tid 4 (0) Success v26) 2026-02-22T23:23:25.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 10 client.4374.objecter have request 4 at 0x7f1c14003060 Op: create unmanaged snap 2026-02-22T23:23:25.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 20 client.4374.objecter waiting for client to reach epoch 26 before calling back 2026-02-22T23:23:25.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 10 client.4374.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:25.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 10 monclient: _renew_subs 2026-02-22T23:23:25.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:25.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=26}) -- 0x7f1c1807f230 con 0x7f1c20002130 2026-02-22T23:23:25.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.211+0000 7f1c297fa640 10 client.4374.objecter done 2026-02-22T23:23:25.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 11 ==== osd_map(26..26 src has 1..26) ==== 626+0+0 (secure 0 0 0) 0x7f1c300993b0 con 0x7f1c20002130 2026-02-22T23:23:25.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c297fa640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_map(26..26 src has 1..26) 2026-02-22T23:23:25.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map got epochs [26,26] > 25 2026-02-22T23:23:25.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map decoding incremental epoch 26 2026-02-22T23:23:25.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c297fa640 20 client.4374.objecter dump_active .. 0 homeless 2026-02-22T23:23:25.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 20 librados: queue_aio_write 0x55708727a510 completion 0x557086fe55f0 write_seq 1 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 10 client.4374.objecter _op_submit op 0x7f1c10006380 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 20 client.4374.objecter _calc_target epoch 26 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 20 client.4374.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 10 client.4374.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 20 client.4374.objecter _get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 10 client.4374.objecter _op_submit oid rbd_info '@2' '@2' [create,write 0~8 in=8b] tid 5 osd.3 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 20 client.4374.objecter get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 15 client.4374.objecter _session_op_assign 3 5 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 15 client.4374.objecter _send_op 5 to 2.2 on osd.3 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26) -- 0x7f1c10006a20 con 0x55708727fb30 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:25.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.217+0000 7f1c2b7fe640 5 client.4374.objecter 1 in flight 2026-02-22T23:23:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 <== osd.3 v2:10.20.193.189:6818/2996878330 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v26'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1c30020070 con 0x55708727fb30 2026-02-22T23:23:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_op_reply(5 rbd_info [create,write 0~8] v26'1 uv1 ondisk = 0) 2026-02-22T23:23:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 10 client.4374.objecter in handle_osd_op_reply 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 7 client.4374.objecter handle_osd_op_reply 5 ondisk uv 1 in 2.2 attempt 0 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 10 client.4374.objecter op 0 rval 0 len 0 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 10 client.4374.objecter op 1 rval 0 len 0 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 15 client.4374.objecter handle_osd_op_reply completed tid 5 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 15 client.4374.objecter _finish_op 5 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 15 client.4374.objecter _session_op_remove 3 5 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 5 client.4374.objecter 0 in flight 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c3906a640 20 librados: complete_aio_write 0x557086fe55f0 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c2affd640 10 client.4374.objecter delete_selfmanaged_snap; pool: 2; snap: 2 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c2affd640 10 client.4374.objecter pool_op_submit 6 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c2affd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:25.457+0000 7f1c2affd640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v26) -- 0x7f1c14004520 con 0x7f1c20002130 2026-02-22T23:23:26.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.097+0000 7f1c1ffff640 10 monclient: tick 2026-02-22T23:23:26.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.097+0000 7f1c1ffff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:26.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.097+0000 7f1c1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:26.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.097+0000 7f1c1ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 12 ==== pool_op_reply(tid 6 (0) Success v27) ==== 43+0+0 (secure 0 0 0) 0x7f1c3004d0d0 con 0x7f1c20002130 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 pool_op_reply(tid 6 (0) Success v27) 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 10 client.4374.objecter handle_pool_op_reply pool_op_reply(tid 6 (0) Success v27) 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 10 client.4374.objecter have request 6 at 0x7f1c14003060 Op: delete unmanaged snap 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 20 client.4374.objecter waiting for client to reach epoch 27 before calling back 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 10 client.4374.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 10 monclient: _renew_subs 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=27}) -- 0x7f1c1807f400 con 0x7f1c20002130 2026-02-22T23:23:26.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.291+0000 7f1c297fa640 10 client.4374.objecter done 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c297fa640 1 -- 10.20.193.6:0/3963142441 <== mon.0 v2:10.20.193.6:3300/0 13 ==== osd_map(27..27 src has 1..27) ==== 654+0+0 (secure 0 0 0) 0x7f1c300605c0 con 0x7f1c20002130 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c297fa640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_map(27..27 src has 1..27) 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map got epochs [27,27] > 26 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c297fa640 3 client.4374.objecter handle_osd_map decoding incremental epoch 27 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c297fa640 20 client.4374.objecter dump_active .. 0 homeless 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 20 librados: queue_aio_write 0x55708727a510 completion 0x7f1c14001d60 write_seq 2 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 10 client.4374.objecter _op_submit op 0x7f1c10006380 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 20 client.4374.objecter _calc_target epoch 27 base rbd_info @2 precalc_pgid 0 pgid 0.0 is_write 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 20 client.4374.objecter _calc_target target rbd_info @2 -> pgid 2.ac0e573a 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 10 client.4374.objecter _calc_target raw pgid 2.ac0e573a -> actual 2.2 acting [3,10] primary 3 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 20 client.4374.objecter _get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 10 client.4374.objecter _op_submit oid rbd_info '@2' '@2' [write 0~19 in=19b] tid 7 osd.3 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 20 client.4374.objecter get_session s=0x55708727f4b0 osd=3 3 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 15 client.4374.objecter _session_op_assign 3 7 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 15 client.4374.objecter _send_op 7 to 2.2 on osd.3 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 1 -- 10.20.193.6:0/3963142441 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x7f1c10006dc0 con 0x55708727fb30 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:26.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.293+0000 7f1c2b7fe640 5 client.4374.objecter 1 in flight 2026-02-22T23:23:26.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 1 -- 10.20.193.6:0/3963142441 <== osd.3 v2:10.20.193.189:6818/2996878330 4 ==== osd_op_reply(7 rbd_info [write 0~19] v27'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1c30020070 con 0x55708727fb30 2026-02-22T23:23:26.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 10 client.4374.objecter ms_dispatch2 0x55708703c9a0 osd_op_reply(7 rbd_info [write 0~19] v27'2 uv2 ondisk = 0) 2026-02-22T23:23:26.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 10 client.4374.objecter in handle_osd_op_reply 2026-02-22T23:23:26.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 7 client.4374.objecter handle_osd_op_reply 7 ondisk uv 2 in 2.2 attempt 0 2026-02-22T23:23:26.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 10 client.4374.objecter op 0 rval 0 len 0 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 15 client.4374.objecter handle_osd_op_reply completed tid 7 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 15 client.4374.objecter _finish_op 7 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 4 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 15 client.4374.objecter _session_op_remove 3 7 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 5 client.4374.objecter 0 in flight 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.395+0000 7f1c3906a640 20 librados: complete_aio_write 0x7f1c14001d60 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 10 librados: watch_flush enter 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 10 librados: watch_flush exit 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 10 client.4374.objecter close_session for osd.3 2026-02-22T23:23:26.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x55708727fb30 msgr2=0x557087281fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x55708727fb30 0x557087281fc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 20 client.4374.objecter put_session s=0x55708727f4b0 osd=3 1 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 10 client.4374.objecter close_session for osd.4 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x557087187900 msgr2=0x5570872017f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x557087187900 0x5570872017f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 20 client.4374.objecter put_session s=0x557086fc1230 osd=4 1 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 20 client.4374.objecter shutdown clearing up homeless session... 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 10 client.4374.objecter successfully canceled tick 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 msgr2=0x7f1c180792e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1c18076e20 0x7f1c180792e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c24000c00 tx=0x7f1c24064000 comp rx=0 tx=0).stop 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 10 monclient: shutdown 2026-02-22T23:23:26.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 msgr2=0x5570872231c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 --2- 10.20.193.6:0/3963142441 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c20002130 0x5570872231c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c610 tx=0x7f1c3000cae0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a2f3640 1 -- 10.20.193.6:0/3963142441 reap_dead start 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a2f3640 1 -- 10.20.193.6:0/3963142441 reap_dead start 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 shutdown_connections 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 >> 10.20.193.6:0/3963142441 conn(0x557087184e50 msgr2=0x557087213990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 shutdown_connections 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 -- 10.20.193.6:0/3963142441 wait complete. 2026-02-22T23:23:26.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.396+0000 7f1c3a5a40c0 1 librados: shutdown 2026-02-22T23:23:26.404 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-22T23:23:26.405 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-22T23:23:26.405 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-22T23:23:26.405 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-22T23:23:26.406 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-22T23:23:26.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.511+0000 7fc40c17e640 10 monclient: get_monmap_and_config 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.511+0000 7fc40c17e640 10 monclient: build_initial_monmap 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.511+0000 7fc40c17e640 10 monclient: monmap: 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:26.512291+0000 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:26.512291+0000 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:26.513 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 1 Processor -- start 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 1 -- start start 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient: init 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient: _add_conns ranks=[5,4,1,0,3,2] 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc4041527d0 0x7fc40406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient(hunting): picked mon.noname-c con 0x7fc4041527d0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc404152bb0 0x7fc4040c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient(hunting): picked mon.noname-b con 0x7fc404152bb0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc4040c5fc0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc40406e300 con 0x7fc4040c5fc0 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc40406f1f0 con 0x7fc404152bb0 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc4040c6470 con 0x7fc4041527d0 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:26.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.512+0000 7fc40c17e640 10 monclient(hunting): authenticate will time out at 1081.239874s 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc4041527d0 0x7fc40406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc40a6f4640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fc4040c5fc0 0x7fc4040c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:53008/0 (socket says 10.20.193.6:53008) 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc40a6f4640 1 -- 10.20.193.6:0/1517542711 learned_addr learned my addr 10.20.193.6:0/1517542711 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 10 monclient(hunting): get_auth_request con 0x7fc4041527d0 auth_method 0 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc4096f2640 1 --2- 10.20.193.6:0/1517542711 >> v2:10.20.193.40:3300/0 conn(0x7fc404152bb0 0x7fc4040c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.513+0000 7fc409ef3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.514+0000 7fc4096f2640 10 monclient(hunting): get_auth_request con 0x7fc404152bb0 auth_method 0 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.514+0000 7fc4096f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.514+0000 7fc4096f2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.514+0000 7fc4096f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 10 monclient(hunting): handle_auth_done global_id 4415 payload 274 2026-02-22T23:23:26.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 1 -- 10.20.193.6:0/1517542711 >> v2:10.20.193.40:3300/0 conn(0x7fc404152bb0 msgr2=0x7fc4040c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:26.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 1 --2- 10.20.193.6:0/1517542711 >> v2:10.20.193.40:3300/0 conn(0x7fc404152bb0 0x7fc4040c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc408ef1640 1 -- 10.20.193.6:0/1517542711 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4091291991 0 0) 0x7fc40406e300 con 0x7fc4040c5fc0 2026-02-22T23:23:26.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 1 -- 10.20.193.6:0/1517542711 >> v1:10.20.193.40:6789/0 conn(0x7fc4040c5fc0 legacy=0x7fc4040c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 10 monclient: _finish_hunting 0 2026-02-22T23:23:26.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 1 monclient: found mon.noname-c 2026-02-22T23:23:26.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:26.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:26.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc409ef3640 1 -- 10.20.193.6:0/1517542711 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40406e5e0 con 0x7fc4041527d0 2026-02-22T23:23:26.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.515+0000 7fc408ef1640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-22T23:23:26.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.516+0000 7fc409ef3640 1 --2- 10.20.193.6:0/1517542711 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc40406da60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc4040caec0 tx=0x7fc3f402ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e2adc83c7752c7 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 1 -- 10.20.193.6:0/1517542711 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f403c070 con 0x7fc4041527d0 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: dump: 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.520 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: _finish_auth 0 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: _check_auth_tickets 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 1 -- 10.20.193.6:0/1517542711 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f402faf0 con 0x7fc4041527d0 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 1 -- 10.20.193.6:0/1517542711 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4037440 con 0x7fc4041527d0 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 5 monclient: authenticate success, global_id 4415 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 4 set_mon_vals no callback set 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc408ef1640 10 monclient: dump: 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.521 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 10 monclient: shutdown 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 -- 10.20.193.6:0/1517542711 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 msgr2=0x7fc40406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 --2- 10.20.193.6:0/1517542711 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc40406da60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc4040caec0 tx=0x7fc3f402ede0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 -- 10.20.193.6:0/1517542711 shutdown_connections 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 --2- 10.20.193.6:0/1517542711 >> v2:10.20.193.40:3300/0 conn(0x7fc404152bb0 0x7fc4040c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 --2- 10.20.193.6:0/1517542711 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc40406da60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 -- 10.20.193.6:0/1517542711 >> 10.20.193.6:0/1517542711 conn(0x7fc4040cd780 msgr2=0x7fc4040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 -- 10.20.193.6:0/1517542711 shutdown_connections 2026-02-22T23:23:26.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.518+0000 7fc40c17e640 1 -- 10.20.193.6:0/1517542711 wait complete. 2026-02-22T23:23:26.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 10 monclient: build_initial_monmap 2026-02-22T23:23:26.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 10 monclient: monmap: 2026-02-22T23:23:26.524 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:26.524 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:26.524 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 1 librados: starting msgr at 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 1 librados: starting objecter 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 1 Processor -- start 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 1 -- start start 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 1 librados: setting wanted keys 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 1 librados: calling monclient init 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.519+0000 7fc40c17e640 10 monclient: init 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4040c5fc0 0x7fc4040ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): picked mon.noname-a con 0x7fc4040c5fc0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc4041517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): picked mon.noname-c con 0x7fc4041527d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:26.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc404152bb0 0x7fc40414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): picked mon.noname-b con 0x7fc404152bb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc40406ee80 con 0x7fc4040c5fc0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc40406eba0 con 0x7fc404152bb0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc40406dfa0 con 0x7fc4041527d0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40c17e640 10 monclient(hunting): authenticate will time out at 1081.247108s 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc4041517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40a6f4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc404152bb0 0x7fc40414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc409ef3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4040c5fc0 0x7fc4040ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc4041517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43252/0 (socket says 10.20.193.6:43252) 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 1 -- 10.20.193.6:0/2864735896 learned_addr learned my addr 10.20.193.6:0/2864735896 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 10 monclient(hunting): get_auth_request con 0x7fc4041527d0 auth_method 0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40a6f4640 10 monclient(hunting): get_auth_request con 0x7fc404152bb0 auth_method 0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40a6f4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40a6f4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc40a6f4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc409ef3640 10 monclient(hunting): get_auth_request con 0x7fc4040c5fc0 auth_method 0 2026-02-22T23:23:26.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc409ef3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc409ef3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc409ef3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.520+0000 7fc4096f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.521+0000 7fc4096f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.521+0000 7fc40a6f4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.521+0000 7fc40a6f4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.521+0000 7fc40a6f4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.521+0000 7fc409ef3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.521+0000 7fc409ef3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.521+0000 7fc409ef3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 10 monclient(hunting): handle_auth_done global_id 4418 payload 824 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 1 -- 10.20.193.6:0/2864735896 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc404152bb0 msgr2=0x7fc40414f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc404152bb0 0x7fc40414f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 1 -- 10.20.193.6:0/2864735896 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4040c5fc0 msgr2=0x7fc4040ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4040c5fc0 0x7fc4040ba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 10 monclient: _finish_hunting 0 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 1 monclient: found mon.noname-c 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:26.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.522+0000 7fc4096f2640 1 -- 10.20.193.6:0/2864735896 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4040ba840 con 0x7fc4041527d0 2026-02-22T23:23:26.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.523+0000 7fc4096f2640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc4041517e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800e810 tx=0x7fc3f800ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:26.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.523+0000 7fc3f27fc640 10 client.?.objecter ms_handle_connect 0x7fc4041527d0 2026-02-22T23:23:26.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.523+0000 7fc3f27fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 1 -- 10.20.193.6:0/2864735896 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8017070 con 0x7fc4041527d0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: dump: 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 1 -- 10.20.193.6:0/2864735896 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f800c8d0 con 0x7fc4041527d0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 5 monclient: authenticate success, global_id 4418 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 10 monclient: _renew_subs 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4040bab10 con 0x7fc4041527d0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 1 -- 10.20.193.6:0/2864735896 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800f040 con 0x7fc4041527d0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f3fff640 4 set_mon_vals no callback set 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc3f27fc640 10 monclient: dump: 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 10 client.4418.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 10 monclient: _renew_subs 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4040bb0d0 con 0x7fc4041527d0 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.525+0000 7fc40c17e640 1 librados: init done 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.526+0000 7fc40c17e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.526+0000 7fc40c17e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.526+0000 7fc40c17e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:26.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.526+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40406da60 con 0x7fc4041527d0 2026-02-22T23:23:26.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.537+0000 7fc3f27fc640 1 -- 10.20.193.6:0/2864735896 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc3f8012050 con 0x7fc4041527d0 2026-02-22T23:23:26.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.538+0000 7fc3f27fc640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc3d4078e00 0x7fc3d407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.538+0000 7fc409ef3640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc3d4078e00 0x7fc3d407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.538+0000 7fc409ef3640 10 monclient: get_auth_request con 0x7fc3d4078e00 auth_method 0 2026-02-22T23:23:26.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.538+0000 7fc3f27fc640 10 client.4418.objecter ms_handle_connect 0x7fc3d4078e00 2026-02-22T23:23:26.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.538+0000 7fc409ef3640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc3d4078e00 0x7fc3d407b2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4039000 tx=0x7fc3f4033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:26.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.633+0000 7fc3f27fc640 1 -- 10.20.193.6:0/2864735896 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fc3f805dae0 con 0x7fc4041527d0 2026-02-22T23:23:26.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.633+0000 7fc3f27fc640 10 client.4418.objecter ms_dispatch2 0x7fc404000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:26.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.633+0000 7fc3f27fc640 3 client.4418.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:26.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.633+0000 7fc3f27fc640 3 client.4418.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:26.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.633+0000 7fc3f27fc640 20 client.4418.objecter dump_active .. 0 homeless 2026-02-22T23:23:26.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.639+0000 7fc3f27fc640 1 -- 10.20.193.6:0/2864735896 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc3f8061aa0 con 0x7fc4041527d0 2026-02-22T23:23:26.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.639+0000 7fc3f27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:26.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.639+0000 7fc3f27fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:26.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.743+0000 7fc40c17e640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2026-02-22T23:23:26.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.743+0000 7fc40c17e640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-22T23:23:26.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.743+0000 7fc40c17e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:26.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.743+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc4040c87c0 con 0x7fc4041527d0 2026-02-22T23:23:26.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.805+0000 7fc3f27fc640 1 -- 10.20.193.6:0/2864735896 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+191896 (secure 0 0 0) 0x7fc3f8065da0 con 0x7fc4041527d0 2026-02-22T23:23:26.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.805+0000 7fc3f27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2026-02-22T23:23:26.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.805+0000 7fc3f27fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:26.808 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 10 librados: watch_flush enter 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 10 librados: watch_flush exit 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 20 client.4418.objecter shutdown clearing up homeless session... 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 10 client.4418.objecter successfully canceled tick 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc3d4078e00 msgr2=0x7fc3d407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc3d4078e00 0x7fc3d407b2c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4039000 tx=0x7fc3f4033000 comp rx=0 tx=0).stop 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 10 monclient: shutdown 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 msgr2=0x7fc4041517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc4041517e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800e810 tx=0x7fc3f800ece0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 shutdown_connections 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc3d4078e00 0x7fc3d407b2c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc404152bb0 0x7fc40414f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4041527d0 0x7fc4041517e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 --2- 10.20.193.6:0/2864735896 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4040c5fc0 0x7fc4040ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 >> 10.20.193.6:0/2864735896 conn(0x7fc4040cd780 msgr2=0x7fc4040c8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:26.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 shutdown_connections 2026-02-22T23:23:26.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 -- 10.20.193.6:0/2864735896 wait complete. 2026-02-22T23:23:26.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.806+0000 7fc40c17e640 1 librados: shutdown 2026-02-22T23:23:26.815 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":5,"flags":0,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6800","nonce":3784423838},{"type":"v1","addr":"10.20.193.189:6801","nonce":3784423838}]},"active_addr":"10.20.193.189:6801/3784423838","active_change":"2026-02-22T23:21:41.654512+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":1950025338}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":1624344828}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":2078959770}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.189:0","nonce":3398618902}]}]} 2026-02-22T23:23:26.818 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-22T23:23:26.818 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-22T23:23:26.818 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.923+0000 7ff7849d3640 10 monclient: get_monmap_and_config 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.923+0000 7ff7849d3640 10 monclient: build_initial_monmap 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.923+0000 7ff7849d3640 10 monclient: monmap: 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:26.924003+0000 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:26.924003+0000 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:26.924 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.923+0000 7ff7849d3640 1 Processor -- start 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.923+0000 7ff7849d3640 1 -- start start 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.923+0000 7ff7849d3640 10 monclient: init 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient: _add_conns ranks=[1,3,5,2,0,4] 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff77c155c70 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff77c15d9d0 0x7ff77c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient(hunting): picked mon.noname-a con 0x7ff77c15d9d0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff77c0ccb30 0x7ff77c15f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient(hunting): picked mon.noname-c con 0x7ff77c0ccb30 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff77c072e90 con 0x7ff77c155c70 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c06f440 con 0x7ff77c15d9d0 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff77c0cbcb0 con 0x7ff77c0ccb30 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:26.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff7849d3640 10 monclient(hunting): authenticate will time out at 1081.651374s 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff781f47640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff77c15d9d0 0x7ff77c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff782f49640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff77c0ccb30 0x7ff77c15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff782748640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7ff77c155c70 0x7ff77c15d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:53010/0 (socket says 10.20.193.6:53010) 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.924+0000 7ff782748640 1 -- 10.20.193.6:0/2235254059 learned_addr learned my addr 10.20.193.6:0/2235254059 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff781f47640 10 monclient(hunting): get_auth_request con 0x7ff77c15d9d0 auth_method 0 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff781f47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff781f47640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff781f47640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff782f49640 10 monclient(hunting): get_auth_request con 0x7ff77c0ccb30 auth_method 0 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff782f49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff782f49640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff782f49640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff781f47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff781f47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff781f47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff782f49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff782f49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.925+0000 7ff782f49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 10 monclient(hunting): handle_auth_done global_id 4383 payload 274 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 1 -- 10.20.193.6:0/2235254059 >> v2:10.20.193.80:3300/0 conn(0x7ff77c0ccb30 msgr2=0x7ff77c15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 1 --2- 10.20.193.6:0/2235254059 >> v2:10.20.193.80:3300/0 conn(0x7ff77c0ccb30 0x7ff77c15f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 1 -- 10.20.193.6:0/2235254059 >> v1:10.20.193.40:6789/0 conn(0x7ff77c155c70 legacy=0x7ff77c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 10 monclient: _finish_hunting 0 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 1 monclient: found mon.noname-a 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:26.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.926+0000 7ff781f47640 1 -- 10.20.193.6:0/2235254059 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c06f690 con 0x7ff77c15d9d0 2026-02-22T23:23:26.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.927+0000 7ff781f47640 1 --2- 10.20.193.6:0/2235254059 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c15d9d0 0x7ff77c15f3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff77c069a40 tx=0x7ff76802ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52f13b5d6b5502f server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 1 -- 10.20.193.6:0/2235254059 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76803c070 con 0x7ff77c15d9d0 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: dump: 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: _finish_auth 0 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: _check_auth_tickets 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 1 -- 10.20.193.6:0/2235254059 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76802fc20 con 0x7ff77c15d9d0 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 1 -- 10.20.193.6:0/2235254059 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768037440 con 0x7ff77c15d9d0 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 5 monclient: authenticate success, global_id 4383 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:26.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 4 set_mon_vals no callback set 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff781746640 10 monclient: dump: 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 10 monclient: shutdown 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 -- 10.20.193.6:0/2235254059 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c15d9d0 msgr2=0x7ff77c15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 --2- 10.20.193.6:0/2235254059 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c15d9d0 0x7ff77c15f3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff77c069a40 tx=0x7ff76802ef10 comp rx=0 tx=0).stop 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 -- 10.20.193.6:0/2235254059 shutdown_connections 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 --2- 10.20.193.6:0/2235254059 >> v2:10.20.193.80:3300/0 conn(0x7ff77c0ccb30 0x7ff77c15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 --2- 10.20.193.6:0/2235254059 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c15d9d0 0x7ff77c15f3c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 -- 10.20.193.6:0/2235254059 >> 10.20.193.6:0/2235254059 conn(0x7ff77c0cc010 msgr2=0x7ff77c0cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 -- 10.20.193.6:0/2235254059 shutdown_connections 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.929+0000 7ff7849d3640 1 -- 10.20.193.6:0/2235254059 wait complete. 2026-02-22T23:23:26.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient: build_initial_monmap 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient: monmap: 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 librados: starting msgr at 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 librados: starting objecter 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 Processor -- start 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 -- start start 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 librados: setting wanted keys 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 librados: calling monclient init 2026-02-22T23:23:26.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient: init 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c0ccb30 0x7ff77c1f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): picked mon.noname-a con 0x7ff77c0ccb30 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff77c155c70 0x7ff77c06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): picked mon.noname-c con 0x7ff77c155c70 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff77c15d9d0 0x7ff77c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): picked mon.noname-b con 0x7ff77c15d9d0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c06ff30 con 0x7ff77c0ccb30 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c06fc50 con 0x7ff77c15d9d0 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c070210 con 0x7ff77c155c70 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.930+0000 7ff7849d3640 10 monclient(hunting): authenticate will time out at 1081.657899s 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c0ccb30 0x7ff77c1f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff77c155c70 0x7ff77c06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff77c15d9d0 0x7ff77c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c0ccb30 0x7ff77c1f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58620/0 (socket says 10.20.193.6:58620) 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 1 -- 10.20.193.6:0/4138716757 learned_addr learned my addr 10.20.193.6:0/4138716757 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 10 monclient(hunting): get_auth_request con 0x7ff77c0ccb30 auth_method 0 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 10 monclient(hunting): get_auth_request con 0x7ff77c155c70 auth_method 0 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 10 monclient(hunting): get_auth_request con 0x7ff77c15d9d0 auth_method 0 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782748640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff781f47640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.931+0000 7ff782f49640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 10 monclient(hunting): handle_auth_done global_id 4386 payload 824 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 1 -- 10.20.193.6:0/4138716757 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff77c155c70 msgr2=0x7ff77c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff77c155c70 0x7ff77c06dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 1 -- 10.20.193.6:0/4138716757 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff77c15d9d0 msgr2=0x7ff77c1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff77c15d9d0 0x7ff77c1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 10 monclient: _finish_hunting 0 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 1 monclient: found mon.noname-a 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:26.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.933+0000 7ff782748640 1 -- 10.20.193.6:0/4138716757 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c06ca30 con 0x7ff77c0ccb30 2026-02-22T23:23:26.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.934+0000 7ff782748640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c0ccb30 0x7ff77c1f6e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff77400c730 tx=0x7ff77400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:26.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.934+0000 7ff766ffd640 10 client.?.objecter ms_handle_connect 0x7ff77c0ccb30 2026-02-22T23:23:26.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.934+0000 7ff766ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:26.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 1 -- 10.20.193.6:0/4138716757 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774017070 con 0x7ff77c0ccb30 2026-02-22T23:23:26.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:26.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: dump: 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:26.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 1 -- 10.20.193.6:0/4138716757 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff774005b90 con 0x7ff77c0ccb30 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 1 -- 10.20.193.6:0/4138716757 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774007070 con 0x7ff77c0ccb30 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff766ffd640 10 monclient: dump: 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:26.940 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff780f45640 4 set_mon_vals no callback set 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 5 monclient: authenticate success, global_id 4386 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 10 monclient: _renew_subs 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c06cd10 con 0x7ff77c0ccb30 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 10 client.4386.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 10 monclient: _renew_subs 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c06d2d0 con 0x7ff77c0ccb30 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.935+0000 7ff7849d3640 1 librados: init done 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.936+0000 7ff7849d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:26.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.936+0000 7ff7849d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:26.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.936+0000 7ff7849d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:26.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.936+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c06d600 con 0x7ff77c0ccb30 2026-02-22T23:23:26.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.947+0000 7ff766ffd640 1 -- 10.20.193.6:0/4138716757 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff774007210 con 0x7ff77c0ccb30 2026-02-22T23:23:26.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.948+0000 7ff766ffd640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff74c078e00 0x7ff74c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:26.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.948+0000 7ff781f47640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff74c078e00 0x7ff74c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:26.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.948+0000 7ff781f47640 10 monclient: get_auth_request con 0x7ff74c078e00 auth_method 0 2026-02-22T23:23:26.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.948+0000 7ff766ffd640 10 client.4386.objecter ms_handle_connect 0x7ff74c078e00 2026-02-22T23:23:26.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.948+0000 7ff781f47640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff74c078e00 0x7ff74c07b2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff768039000 tx=0x7ff768033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:26.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.949+0000 7ff766ffd640 1 -- 10.20.193.6:0/4138716757 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7ff77405d480 con 0x7ff77c0ccb30 2026-02-22T23:23:26.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.949+0000 7ff766ffd640 10 client.4386.objecter ms_dispatch2 0x7ff77c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:26.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.949+0000 7ff766ffd640 3 client.4386.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:26.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.949+0000 7ff766ffd640 3 client.4386.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:26.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:26.949+0000 7ff766ffd640 20 client.4386.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.045+0000 7ff766ffd640 1 -- 10.20.193.6:0/4138716757 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff774021020 con 0x7ff77c0ccb30 2026-02-22T23:23:27.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.045+0000 7ff766ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.045+0000 7ff766ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:27.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.129+0000 7ff7849d3640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:27.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.129+0000 7ff7849d3640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:27.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.129+0000 7ff7849d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.129+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff77c074850 con 0x7ff77c0ccb30 2026-02-22T23:23:27.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.178+0000 7ff766ffd640 1 -- 10.20.193.6:0/4138716757 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+24623 (secure 0 0 0) 0x7ff774061440 con 0x7ff77c0ccb30 2026-02-22T23:23:27.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.178+0000 7ff766ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:27.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.178+0000 7ff766ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:27.181 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:27.181 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":27,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:23:26.182967+0000","last_up_change":"2026-02-22T23:23:21.874564+0000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:23:19.944772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T23:23:22.821753+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6824","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6825","nonce":1913492598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6826","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6827","nonce":1913492598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6830","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6831","nonce":1913492598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6828","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6829","nonce":1913492598}]},"public_addr":"10.20.193.6:6825/1913492598","cluster_addr":"10.20.193.6:6827/1913492598","heartbeat_back_addr":"10.20.193.6:6831/1913492598","heartbeat_front_addr":"10.20.193.6:6829/1913492598","state":["exists","up"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6824","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6825","nonce":2813004313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6826","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6827","nonce":2813004313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6830","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6831","nonce":2813004313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6828","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6829","nonce":2813004313}]},"public_addr":"10.20.193.40:6825/2813004313","cluster_addr":"10.20.193.40:6827/2813004313","heartbeat_back_addr":"10.20.193.40:6831/2813004313","heartbeat_front_addr":"10.20.193.40:6829/2813004313","state":["exists","up"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6817","nonce":1223585399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6818","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6819","nonce":1223585399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6822","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6823","nonce":1223585399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6821","nonce":1223585399}]},"public_addr":"10.20.193.80:6817/1223585399","cluster_addr":"10.20.193.80:6819/1223585399","heartbeat_back_addr":"10.20.193.80:6823/1223585399","heartbeat_front_addr":"10.20.193.80:6821/1223585399","state":["exists","up"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6818","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6819","nonce":2996878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6820","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6821","nonce":2996878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6824","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6825","nonce":2996878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6822","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6823","nonce":2996878330}]},"public_addr":"10.20.193.189:6819/2996878330","cluster_addr":"10.20.193.189:6821/2996878330","heartbeat_back_addr":"10.20.193.189:6825/2996878330","heartbeat_front_addr":"10.20.193.189:6823/2996878330","state":["exists","up"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6800","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6801","nonce":2180113666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6802","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6803","nonce":2180113666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6806","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6807","nonce":2180113666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6804","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6805","nonce":2180113666}]},"public_addr":"10.20.193.6:6801/2180113666","cluster_addr":"10.20.193.6:6803/2180113666","heartbeat_back_addr":"10.20.193.6:6807/2180113666","heartbeat_front_addr":"10.20.193.6:6805/2180113666","state":["exists","up"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6800","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6801","nonce":2956321569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6802","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6803","nonce":2956321569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6806","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6807","nonce":2956321569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6804","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6805","nonce":2956321569}]},"public_addr":"10.20.193.40:6801/2956321569","cluster_addr":"10.20.193.40:6803/2956321569","heartbeat_back_addr":"10.20.193.40:6807/2956321569","heartbeat_front_addr":"10.20.193.40:6805/2956321569","state":["exists","up"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6801","nonce":144861518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6803","nonce":144861518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6807","nonce":144861518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6805","nonce":144861518}]},"public_addr":"10.20.193.80:6801/144861518","cluster_addr":"10.20.193.80:6803/144861518","heartbeat_back_addr":"10.20.193.80:6807/144861518","heartbeat_front_addr":"10.20.193.80:6805/144861518","state":["exists","up"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6810","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6811","nonce":980553521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6812","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6813","nonce":980553521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6816","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6817","nonce":980553521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6814","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6815","nonce":980553521}]},"public_addr":"10.20.193.189:6811/980553521","cluster_addr":"10.20.193.189:6813/980553521","heartbeat_back_addr":"10.20.193.189:6817/980553521","heartbeat_front_addr":"10.20.193.189:6815/980553521","state":["exists","up"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6816","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6817","nonce":1002652796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6818","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6819","nonce":1002652796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6822","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6823","nonce":1002652796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6820","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6821","nonce":1002652796}]},"public_addr":"10.20.193.6:6817/1002652796","cluster_addr":"10.20.193.6:6819/1002652796","heartbeat_back_addr":"10.20.193.6:6823/1002652796","heartbeat_front_addr":"10.20.193.6:6821/1002652796","state":["exists","up"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6808","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6809","nonce":979607499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6810","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6811","nonce":979607499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6814","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6815","nonce":979607499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6812","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6813","nonce":979607499}]},"public_addr":"10.20.193.40:6809/979607499","cluster_addr":"10.20.193.40:6811/979607499","heartbeat_back_addr":"10.20.193.40:6815/979607499","heartbeat_front_addr":"10.20.193.40:6813/979607499","state":["exists","up"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6809","nonce":2974128537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6810","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6811","nonce":2974128537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6814","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6815","nonce":2974128537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6813","nonce":2974128537}]},"public_addr":"10.20.193.80:6809/2974128537","cluster_addr":"10.20.193.80:6811/2974128537","heartbeat_back_addr":"10.20.193.80:6815/2974128537","heartbeat_front_addr":"10.20.193.80:6813/2974128537","state":["exists","up"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6802","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6803","nonce":2070530833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6804","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6805","nonce":2070530833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6808","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6809","nonce":2070530833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6806","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6807","nonce":2070530833}]},"public_addr":"10.20.193.189:6803/2070530833","cluster_addr":"10.20.193.189:6805/2070530833","heartbeat_back_addr":"10.20.193.189:6809/2070530833","heartbeat_front_addr":"10.20.193.189:6807/2070530833","state":["exists","up"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6808","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6809","nonce":138679770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6810","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6811","nonce":138679770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6814","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6815","nonce":138679770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6812","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6813","nonce":138679770}]},"public_addr":"10.20.193.6:6809/138679770","cluster_addr":"10.20.193.6:6811/138679770","heartbeat_back_addr":"10.20.193.6:6815/138679770","heartbeat_front_addr":"10.20.193.6:6813/138679770","state":["exists","up"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6816","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6817","nonce":1944897191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6818","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6819","nonce":1944897191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6822","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6823","nonce":1944897191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6820","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6821","nonce":1944897191}]},"public_addr":"10.20.193.40:6817/1944897191","cluster_addr":"10.20.193.40:6819/1944897191","heartbeat_back_addr":"10.20.193.40:6823/1944897191","heartbeat_front_addr":"10.20.193.40:6821/1944897191","state":["exists","up"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6824","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6825","nonce":3509265050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6826","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6827","nonce":3509265050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6830","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6831","nonce":3509265050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6828","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6829","nonce":3509265050}]},"public_addr":"10.20.193.80:6825/3509265050","cluster_addr":"10.20.193.80:6827/3509265050","heartbeat_back_addr":"10.20.193.80:6831/3509265050","heartbeat_front_addr":"10.20.193.80:6829/3509265050","state":["exists","up"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6826","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6827","nonce":3776650816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6828","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6829","nonce":3776650816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6832","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6833","nonce":3776650816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6830","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6831","nonce":3776650816}]},"public_addr":"10.20.193.189:6827/3776650816","cluster_addr":"10.20.193.189:6829/3776650816","heartbeat_back_addr":"10.20.193.189:6833/3776650816","heartbeat_front_addr":"10.20.193.189:6831/3776650816","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.797315+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.477029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.595886+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.915624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.060069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.938930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.614574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.167676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.905706+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.437161+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.930818+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.151987+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.520133+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.819151+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.612032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.945924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:23:27.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 10 librados: watch_flush enter 2026-02-22T23:23:27.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 10 librados: watch_flush exit 2026-02-22T23:23:27.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 20 client.4386.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 10 client.4386.objecter successfully canceled tick 2026-02-22T23:23:27.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff74c078e00 msgr2=0x7ff74c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff74c078e00 0x7ff74c07b2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff768039000 tx=0x7ff768033000 comp rx=0 tx=0).stop 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 10 monclient: shutdown 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c0ccb30 msgr2=0x7ff77c1f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c0ccb30 0x7ff77c1f6e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff77400c730 tx=0x7ff77400cc00 comp rx=0 tx=0).stop 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 shutdown_connections 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff74c078e00 0x7ff74c07b2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff77c15d9d0 0x7ff77c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff77c155c70 0x7ff77c06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 --2- 10.20.193.6:0/4138716757 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff77c0ccb30 0x7ff77c1f6e20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 >> 10.20.193.6:0/4138716757 conn(0x7ff77c0cc010 msgr2=0x7ff77c0ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 shutdown_connections 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 -- 10.20.193.6:0/4138716757 wait complete. 2026-02-22T23:23:27.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.180+0000 7ff7849d3640 1 librados: shutdown 2026-02-22T23:23:27.191 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-22T23:23:27.191 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:23:27.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.258+0000 7f11fe6bd640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.258+0000 7f11fe6bd640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.258+0000 7f11fe6bd640 10 monclient: monmap: 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.259357+0000 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.259357+0000 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.261 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.262 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.262 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.262 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.259+0000 7f11fe6bd640 1 Processor -- start 2026-02-22T23:23:27.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.259+0000 7f11fe6bd640 1 -- start start 2026-02-22T23:23:27.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.259+0000 7f11fe6bd640 10 monclient: init 2026-02-22T23:23:27.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.259+0000 7f11fe6bd640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.259+0000 7f11fe6bd640 10 monclient: _add_conns ranks=[2,0,3,4,5,1] 2026-02-22T23:23:27.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f11f81527f0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f11f8073530 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f11f815fad0 0x7f11f80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 10 monclient(hunting): picked mon.noname-a con 0x7f11f815fad0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11f80739a0 con 0x7f11f8073530 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11f815d9d0 con 0x7f11f81527f0 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f11f8074b10 con 0x7f11f815fad0 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fe6bd640 10 monclient(hunting): authenticate will time out at 1081.987059s 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fcc33640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f11f815fad0 0x7f11f80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11f77fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f11f8073530 0x7f11f815f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58750/0 (socket says 10.20.193.6:58750) 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11f77fe640 1 -- 10.20.193.6:0/3808109882 learned_addr learned my addr 10.20.193.6:0/3808109882 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fcc33640 10 monclient(hunting): get_auth_request con 0x7f11f815fad0 auth_method 0 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fcc33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fcc33640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.260+0000 7f11fcc33640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.261+0000 7f11fcc33640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.261+0000 7f11fcc33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.261+0000 7f11fcc33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 10 monclient(hunting): handle_auth_done global_id 4389 payload 274 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 1 -- 10.20.193.6:0/3808109882 >> v1:10.20.193.80:6789/0 conn(0x7f11f81527f0 legacy=0x7f11f815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 1 -- 10.20.193.6:0/3808109882 >> v1:10.20.193.6:6789/0 conn(0x7f11f8073530 legacy=0x7f11f815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.262+0000 7f11fcc33640 1 -- 10.20.193.6:0/3808109882 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f8073c20 con 0x7f11f815fad0 2026-02-22T23:23:27.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.265+0000 7f11fcc33640 1 --2- 10.20.193.6:0/3808109882 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f815fad0 0x7f11f80ccf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009d90 tx=0x7f11ec02f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2402e105e22864d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.268+0000 7f11f6ffd640 1 -- 10.20.193.6:0/3808109882 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec03c070 con 0x7f11f815fad0 2026-02-22T23:23:27.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.268+0000 7f11f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.268+0000 7f11f6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.268+0000 7f11f6ffd640 10 monclient: dump: 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.268+0000 7f11f6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.268+0000 7f11f6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.268+0000 7f11f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.269+0000 7f11fe6bd640 5 monclient: authenticate success, global_id 4389 2026-02-22T23:23:27.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.269+0000 7f11fe6bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11f6ffd640 1 -- 10.20.193.6:0/3808109882 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ec037440 con 0x7f11f815fad0 2026-02-22T23:23:27.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11f6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11f6ffd640 1 -- 10.20.193.6:0/3808109882 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec037810 con 0x7f11f815fad0 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11f6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11f6ffd640 10 monclient: dump: 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.276 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 4 set_mon_vals no callback set 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 10 monclient: shutdown 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 1 -- 10.20.193.6:0/3808109882 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f815fad0 msgr2=0x7f11f80ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/3808109882 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f815fad0 0x7f11f80ccf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009d90 tx=0x7f11ec02f1e0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 1 -- 10.20.193.6:0/3808109882 shutdown_connections 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/3808109882 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f815fad0 0x7f11f80ccf70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 1 -- 10.20.193.6:0/3808109882 >> 10.20.193.6:0/3808109882 conn(0x7f11f80ce730 msgr2=0x7f11f80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 1 -- 10.20.193.6:0/3808109882 shutdown_connections 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.270+0000 7f11fe6bd640 1 -- 10.20.193.6:0/3808109882 wait complete. 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 10 monclient: monmap: 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.278 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 1 librados: starting msgr at 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 1 librados: starting objecter 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 1 Processor -- start 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 1 -- start start 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 1 librados: setting wanted keys 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 1 librados: calling monclient init 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.271+0000 7f11fe6bd640 10 monclient: init 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11f8073530 0x7f11f81484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): picked mon.noname-b con 0x7f11f8073530 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f81527f0 0x7f11f8157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): picked mon.noname-a con 0x7f11f81527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f11f815fad0 0x7f11f8157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): picked mon.noname-c con 0x7f11f815fad0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f11f80744c0 con 0x7f11f81527f0 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f11f80741e0 con 0x7f11f8073530 2026-02-22T23:23:27.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f11f80747a0 con 0x7f11f815fad0 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.272+0000 7f11fe6bd640 10 monclient(hunting): authenticate will time out at 1081.999199s 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11fcc33640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f11f815fad0 0x7f11f8157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11f7fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11f8073530 0x7f11f81484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11f77fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f81527f0 0x7f11f8157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11fcc33640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f11f815fad0 0x7f11f8157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43278/0 (socket says 10.20.193.6:43278) 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11fcc33640 1 -- 10.20.193.6:0/114810306 learned_addr learned my addr 10.20.193.6:0/114810306 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11fcc33640 10 monclient(hunting): get_auth_request con 0x7f11f815fad0 auth_method 0 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11fcc33640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11fcc33640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11fcc33640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11f7fff640 10 monclient(hunting): get_auth_request con 0x7f11f8073530 auth_method 0 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.273+0000 7f11f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.274+0000 7f11f77fe640 10 monclient(hunting): get_auth_request con 0x7f11f81527f0 auth_method 0 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.274+0000 7f11f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.274+0000 7f11f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.274+0000 7f11f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.275+0000 7f11f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.275+0000 7f11f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.275+0000 7f11f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.276+0000 7f11f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.276+0000 7f11f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.276+0000 7f11f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.276+0000 7f11fcc33640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.276+0000 7f11fcc33640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.276+0000 7f11fcc33640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 10 monclient(hunting): handle_auth_done global_id 4392 payload 824 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 1 -- 10.20.193.6:0/114810306 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f11f815fad0 msgr2=0x7f11f8157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f11f815fad0 0x7f11f8157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 1 -- 10.20.193.6:0/114810306 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11f8073530 msgr2=0x7f11f81484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11f8073530 0x7f11f81484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.277+0000 7f11f77fe640 1 -- 10.20.193.6:0/114810306 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f8148f50 con 0x7f11f81527f0 2026-02-22T23:23:27.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.278+0000 7f11f77fe640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f81527f0 0x7f11f8157760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c530 tx=0x7f11e800ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.278+0000 7f11f4ff9640 10 client.?.objecter ms_handle_connect 0x7f11f81527f0 2026-02-22T23:23:27.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.278+0000 7f11f4ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 1 -- 10.20.193.6:0/114810306 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8017070 con 0x7f11f81527f0 2026-02-22T23:23:27.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: dump: 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.310 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 5 monclient: authenticate success, global_id 4392 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 1 -- 10.20.193.6:0/114810306 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e8005de0 con 0x7f11f81527f0 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 1 -- 10.20.193.6:0/114810306 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e80072b0 con 0x7f11f81527f0 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f67fc640 4 set_mon_vals no callback set 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11f4ff9640 10 monclient: dump: 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 10 monclient: _renew_subs 2026-02-22T23:23:27.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f81f66b0 con 0x7f11f81527f0 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 10 client.4392.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 10 monclient: _renew_subs 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f81569a0 con 0x7f11f81527f0 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.307+0000 7f11fe6bd640 1 librados: init done 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.308+0000 7f11fe6bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.308+0000 7f11fe6bd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.308+0000 7f11fe6bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.308+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f8074f80 con 0x7f11f81527f0 2026-02-22T23:23:27.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.409+0000 7f11f4ff9640 1 -- 10.20.193.6:0/114810306 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f11e8007450 con 0x7f11f81527f0 2026-02-22T23:23:27.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.410+0000 7f11f4ff9640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f11c8078e50 0x7f11c807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.410+0000 7f11f7fff640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f11c8078e50 0x7f11c807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.410+0000 7f11f7fff640 10 monclient: get_auth_request con 0x7f11c8078e50 auth_method 0 2026-02-22T23:23:27.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.411+0000 7f11f7fff640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f11c8078e50 0x7f11c807b310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11e40046e0 tx=0x7f11e40091c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.411+0000 7f11f4ff9640 10 client.4392.objecter ms_handle_connect 0x7f11c8078e50 2026-02-22T23:23:27.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 1 -- 10.20.193.6:0/114810306 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f11e8007770 con 0x7f11f81527f0 2026-02-22T23:23:27.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 10 client.4392.objecter ms_dispatch2 0x7f11f8000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 3 client.4392.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 3 client.4392.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 20 client.4392.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 1 -- 10.20.193.6:0/114810306 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f11e8021020 con 0x7f11f81527f0 2026-02-22T23:23:27.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.417+0000 7f11f4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.501+0000 7f11fe6bd640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.501+0000 7f11fe6bd640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.501+0000 7f11fe6bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.501+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11f8151700 con 0x7f11f81527f0 2026-02-22T23:23:27.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.521+0000 7f11f4ff9640 1 -- 10.20.193.6:0/114810306 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+24623 (secure 0 0 0) 0x7f11f80739a0 con 0x7f11f81527f0 2026-02-22T23:23:27.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.521+0000 7f11f4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:23:27.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.521+0000 7f11f4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:27.522 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:27.523 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":27,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:23:26.182967+0000","last_up_change":"2026-02-22T23:23:21.874564+0000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:23:19.944772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T23:23:22.821753+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6824","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6825","nonce":1913492598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6826","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6827","nonce":1913492598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6830","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6831","nonce":1913492598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6828","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6829","nonce":1913492598}]},"public_addr":"10.20.193.6:6825/1913492598","cluster_addr":"10.20.193.6:6827/1913492598","heartbeat_back_addr":"10.20.193.6:6831/1913492598","heartbeat_front_addr":"10.20.193.6:6829/1913492598","state":["exists","up"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6824","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6825","nonce":2813004313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6826","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6827","nonce":2813004313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6830","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6831","nonce":2813004313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6828","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6829","nonce":2813004313}]},"public_addr":"10.20.193.40:6825/2813004313","cluster_addr":"10.20.193.40:6827/2813004313","heartbeat_back_addr":"10.20.193.40:6831/2813004313","heartbeat_front_addr":"10.20.193.40:6829/2813004313","state":["exists","up"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6817","nonce":1223585399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6818","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6819","nonce":1223585399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6822","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6823","nonce":1223585399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6821","nonce":1223585399}]},"public_addr":"10.20.193.80:6817/1223585399","cluster_addr":"10.20.193.80:6819/1223585399","heartbeat_back_addr":"10.20.193.80:6823/1223585399","heartbeat_front_addr":"10.20.193.80:6821/1223585399","state":["exists","up"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6818","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6819","nonce":2996878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6820","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6821","nonce":2996878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6824","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6825","nonce":2996878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6822","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6823","nonce":2996878330}]},"public_addr":"10.20.193.189:6819/2996878330","cluster_addr":"10.20.193.189:6821/2996878330","heartbeat_back_addr":"10.20.193.189:6825/2996878330","heartbeat_front_addr":"10.20.193.189:6823/2996878330","state":["exists","up"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6800","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6801","nonce":2180113666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6802","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6803","nonce":2180113666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6806","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6807","nonce":2180113666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6804","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6805","nonce":2180113666}]},"public_addr":"10.20.193.6:6801/2180113666","cluster_addr":"10.20.193.6:6803/2180113666","heartbeat_back_addr":"10.20.193.6:6807/2180113666","heartbeat_front_addr":"10.20.193.6:6805/2180113666","state":["exists","up"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6800","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6801","nonce":2956321569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6802","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6803","nonce":2956321569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6806","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6807","nonce":2956321569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6804","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6805","nonce":2956321569}]},"public_addr":"10.20.193.40:6801/2956321569","cluster_addr":"10.20.193.40:6803/2956321569","heartbeat_back_addr":"10.20.193.40:6807/2956321569","heartbeat_front_addr":"10.20.193.40:6805/2956321569","state":["exists","up"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6801","nonce":144861518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6803","nonce":144861518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6807","nonce":144861518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6805","nonce":144861518}]},"public_addr":"10.20.193.80:6801/144861518","cluster_addr":"10.20.193.80:6803/144861518","heartbeat_back_addr":"10.20.193.80:6807/144861518","heartbeat_front_addr":"10.20.193.80:6805/144861518","state":["exists","up"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6810","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6811","nonce":980553521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6812","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6813","nonce":980553521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6816","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6817","nonce":980553521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6814","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6815","nonce":980553521}]},"public_addr":"10.20.193.189:6811/980553521","cluster_addr":"10.20.193.189:6813/980553521","heartbeat_back_addr":"10.20.193.189:6817/980553521","heartbeat_front_addr":"10.20.193.189:6815/980553521","state":["exists","up"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6816","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6817","nonce":1002652796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6818","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6819","nonce":1002652796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6822","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6823","nonce":1002652796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6820","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6821","nonce":1002652796}]},"public_addr":"10.20.193.6:6817/1002652796","cluster_addr":"10.20.193.6:6819/1002652796","heartbeat_back_addr":"10.20.193.6:6823/1002652796","heartbeat_front_addr":"10.20.193.6:6821/1002652796","state":["exists","up"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6808","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6809","nonce":979607499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6810","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6811","nonce":979607499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6814","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6815","nonce":979607499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6812","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6813","nonce":979607499}]},"public_addr":"10.20.193.40:6809/979607499","cluster_addr":"10.20.193.40:6811/979607499","heartbeat_back_addr":"10.20.193.40:6815/979607499","heartbeat_front_addr":"10.20.193.40:6813/979607499","state":["exists","up"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6809","nonce":2974128537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6810","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6811","nonce":2974128537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6814","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6815","nonce":2974128537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6813","nonce":2974128537}]},"public_addr":"10.20.193.80:6809/2974128537","cluster_addr":"10.20.193.80:6811/2974128537","heartbeat_back_addr":"10.20.193.80:6815/2974128537","heartbeat_front_addr":"10.20.193.80:6813/2974128537","state":["exists","up"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6802","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6803","nonce":2070530833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6804","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6805","nonce":2070530833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6808","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6809","nonce":2070530833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6806","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6807","nonce":2070530833}]},"public_addr":"10.20.193.189:6803/2070530833","cluster_addr":"10.20.193.189:6805/2070530833","heartbeat_back_addr":"10.20.193.189:6809/2070530833","heartbeat_front_addr":"10.20.193.189:6807/2070530833","state":["exists","up"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6808","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6809","nonce":138679770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6810","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6811","nonce":138679770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6814","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6815","nonce":138679770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6812","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6813","nonce":138679770}]},"public_addr":"10.20.193.6:6809/138679770","cluster_addr":"10.20.193.6:6811/138679770","heartbeat_back_addr":"10.20.193.6:6815/138679770","heartbeat_front_addr":"10.20.193.6:6813/138679770","state":["exists","up"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6816","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6817","nonce":1944897191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6818","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6819","nonce":1944897191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6822","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6823","nonce":1944897191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6820","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6821","nonce":1944897191}]},"public_addr":"10.20.193.40:6817/1944897191","cluster_addr":"10.20.193.40:6819/1944897191","heartbeat_back_addr":"10.20.193.40:6823/1944897191","heartbeat_front_addr":"10.20.193.40:6821/1944897191","state":["exists","up"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6824","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6825","nonce":3509265050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6826","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6827","nonce":3509265050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6830","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6831","nonce":3509265050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6828","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6829","nonce":3509265050}]},"public_addr":"10.20.193.80:6825/3509265050","cluster_addr":"10.20.193.80:6827/3509265050","heartbeat_back_addr":"10.20.193.80:6831/3509265050","heartbeat_front_addr":"10.20.193.80:6829/3509265050","state":["exists","up"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6826","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6827","nonce":3776650816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6828","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6829","nonce":3776650816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6832","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6833","nonce":3776650816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6830","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6831","nonce":3776650816}]},"public_addr":"10.20.193.189:6827/3776650816","cluster_addr":"10.20.193.189:6829/3776650816","heartbeat_back_addr":"10.20.193.189:6833/3776650816","heartbeat_front_addr":"10.20.193.189:6831/3776650816","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.797315+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.477029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.595886+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.915624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.060069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.938930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.614574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.167676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.905706+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.437161+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.930818+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.151987+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.520133+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.819151+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.612032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.945924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 10 librados: watch_flush enter 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 10 librados: watch_flush exit 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 20 client.4392.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 10 client.4392.objecter successfully canceled tick 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f11c8078e50 msgr2=0x7f11c807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f11c8078e50 0x7f11c807b310 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f11e40046e0 tx=0x7f11e40091c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 10 monclient: shutdown 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f81527f0 msgr2=0x7f11f8157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f81527f0 0x7f11f8157760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f11e800c530 tx=0x7f11e800ca00 comp rx=0 tx=0).stop 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 shutdown_connections 2026-02-22T23:23:27.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f11c8078e50 0x7f11c807b310 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f11f815fad0 0x7f11f8157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11f81527f0 0x7f11f8157760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 --2- 10.20.193.6:0/114810306 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11f8073530 0x7f11f81484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 >> 10.20.193.6:0/114810306 conn(0x7f11f80ce730 msgr2=0x7f11f81519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 shutdown_connections 2026-02-22T23:23:27.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 -- 10.20.193.6:0/114810306 wait complete. 2026-02-22T23:23:27.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.522+0000 7f11fe6bd640 1 librados: shutdown 2026-02-22T23:23:27.532 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T23:23:27.532 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T23:23:27.532 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2026-02-22T23:23:27.533 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2026-02-22T23:23:27.534 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2026-02-22T23:23:27.534 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2026-02-22T23:23:27.534 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2026-02-22T23:23:27.534 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2026-02-22T23:23:27.534 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2026-02-22T23:23:27.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.613+0000 7f85effd3640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.613+0000 7f85effd3640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.613+0000 7f85effd3640 10 monclient: monmap: 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.614255+0000 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.614255+0000 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 1 Processor -- start 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 1 -- start start 2026-02-22T23:23:27.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient: init 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient: _add_conns ranks=[1,5,4,3,0,2] 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f85e8155f80 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f85e815da30 0x7f85e815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient(hunting): picked mon.noname-c con 0x7f85e815da30 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f85e80ccb40 0x7f85e80caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient(hunting): picked mon.noname-b con 0x7f85e80ccb40 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e80d0380 con 0x7f85e8155f80 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85e806d5b0 con 0x7f85e80ccb40 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f85e806c9a0 con 0x7f85e815da30 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.614+0000 7f85effd3640 10 monclient(hunting): authenticate will time out at 1082.341929s 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f85e815da30 0x7f85e815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ee549640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f85e80ccb40 0x7f85e80caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f85e815da30 0x7f85e815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43286/0 (socket says 10.20.193.6:43286) 2026-02-22T23:23:27.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 1 -- 10.20.193.6:0/3654791709 learned_addr learned my addr 10.20.193.6:0/3654791709 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 10 monclient(hunting): get_auth_request con 0x7f85e815da30 auth_method 0 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ee549640 10 monclient(hunting): get_auth_request con 0x7f85e80ccb40 auth_method 0 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ee549640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ee549640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ee549640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7f85ed547640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7fcce4f1b640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7fcce4f1b640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.615+0000 7fcce4f1b640 10 monclient: monmap: 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.616602+0000 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.616602+0000 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ee549640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ee549640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ee549640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7fcce4f1b640 1 Processor -- start 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7fcce4f1b640 1 -- start start 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7fcce4f1b640 10 monclient: init 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 10 monclient(hunting): handle_auth_done global_id 4430 payload 274 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 1 -- 10.20.193.6:0/3654791709 >> v2:10.20.193.40:3300/0 conn(0x7f85e80ccb40 msgr2=0x7f85e80caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 1 --2- 10.20.193.6:0/3654791709 >> v2:10.20.193.40:3300/0 conn(0x7f85e80ccb40 0x7f85e80caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 1 -- 10.20.193.6:0/3654791709 >> v1:10.20.193.40:6789/0 conn(0x7f85e8155f80 legacy=0x7f85e815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.616+0000 7f85ed547640 1 -- 10.20.193.6:0/3654791709 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e806cc80 con 0x7f85e815da30 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient: _add_conns ranks=[2,4,3,5,1,0] 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcce0155e70 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fcce0074d60 0x7fcce00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient(hunting): picked mon.noname-b con 0x7fcce0074d60 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcce0156250 0x7fcce015d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient(hunting): picked mon.noname-a con 0x7fcce0156250 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce00ccb50 con 0x7fcce0155e70 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcce00cce30 con 0x7fcce0156250 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcce00cd110 con 0x7fcce0074d60 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.617+0000 7fcce4f1b640 10 monclient(hunting): authenticate will time out at 1082.344281s 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccde575640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fcce0155e70 0x7fcce0074120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:48106/0 (socket says 10.20.193.6:48106) 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccde575640 1 -- 10.20.193.6:0/3661876388 learned_addr learned my addr 10.20.193.6:0/3661876388 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7f85ed547640 1 --2- 10.20.193.6:0/3654791709 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e815da30 0x7f85e815c880 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85e815f380 tx=0x7f85dc02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c8b6703762b22e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccded76640 1 --2- 10.20.193.6:0/3661876388 >> v2:10.20.193.6:3300/0 conn(0x7fcce0156250 0x7fcce015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccddd74640 1 --2- 10.20.193.6:0/3661876388 >> v2:10.20.193.40:3300/0 conn(0x7fcce0074d60 0x7fcce00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccddd74640 10 monclient(hunting): get_auth_request con 0x7fcce0074d60 auth_method 0 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccddd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccddd74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccddd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccded76640 10 monclient(hunting): get_auth_request con 0x7fcce0156250 auth_method 0 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccded76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccded76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7fccded76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7f8b86f5c640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7f8b86f5c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.618+0000 7f8b86f5c640 10 monclient: monmap: 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.619536+0000 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.619536+0000 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7fccded76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7fccded76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7fccded76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7f8b86f5c640 1 Processor -- start 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7f8b86f5c640 1 -- start start 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7f8b86f5c640 10 monclient: init 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7fccddd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7fccddd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.619+0000 7fccddd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient: _add_conns ranks=[4,1,5,0,2,3] 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f5f8e144640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f5f8e144640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f5f8e144640 10 monclient: monmap: 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.620801+0000 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.620801+0000 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.622 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8b80155ef0 0x7f8b8015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient(hunting): picked mon.noname-b con 0x7f8b80155ef0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8b801562d0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8b800cde80 0x7f8b800c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient(hunting): picked mon.noname-c con 0x7f8b800cde80 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b800ce3b0 con 0x7f8b801562d0 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8b8015cdb0 con 0x7f8b80155ef0 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8b800d2730 con 0x7f8b800cde80 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85ecd46640 1 -- 10.20.193.6:0/3654791709 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc03c070 con 0x7f85e815da30 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85ecd46640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f8b86f5c640 10 monclient(hunting): authenticate will time out at 1082.347469s 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85ecd46640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85ecd46640 10 monclient: dump: 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85ecd46640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85ecd46640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85ecd46640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85effd3640 5 monclient: authenticate success, global_id 4430 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f85effd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 10 monclient(hunting): handle_auth_done global_id 4414 payload 274 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 1 -- 10.20.193.6:0/3661876388 >> v2:10.20.193.6:3300/0 conn(0x7fcce0156250 msgr2=0x7fcce015d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 1 --2- 10.20.193.6:0/3661876388 >> v2:10.20.193.6:3300/0 conn(0x7fcce0156250 0x7fcce015d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 1 -- 10.20.193.6:0/3661876388 >> v1:10.20.193.80:6789/0 conn(0x7fcce0155e70 legacy=0x7fcce0074120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7fccddd74640 1 -- 10.20.193.6:0/3661876388 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce00cd3f0 con 0x7fcce0074d60 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f5f8e144640 1 Processor -- start 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f5f8e144640 1 -- start start 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.620+0000 7f5f8e144640 10 monclient: init 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85ecd46640 1 -- 10.20.193.6:0/3654791709 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85dc02fb30 con 0x7f85e815da30 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85ecd46640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85ecd46640 1 -- 10.20.193.6:0/3654791709 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc037440 con 0x7f85e815da30 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85effd3640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85effd3640 4 set_mon_vals no callback set 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85ecd46640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85ecd46640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b84cd1640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8b80155ef0 0x7f8b8015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85ecd46640 10 monclient: dump: 2026-02-22T23:23:27.625 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b84cd1640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f8b80155ef0 0x7f8b8015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:32992/0 (socket says 10.20.193.6:32992) 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b84cd1640 1 -- 10.20.193.6:0/1079165846 learned_addr learned my addr 10.20.193.6:0/1079165846 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b84cd1640 10 monclient(hunting): get_auth_request con 0x7f8b80155ef0 auth_method 0 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b84cd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b84cd1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b84cd1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7fe611ed3640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7fe611ed3640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7fe611ed3640 10 monclient: monmap: 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.622115+0000 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.622115+0000 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.626 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f8b854d2640 1 --2- 10.20.193.6:0/1079165846 >> v2:10.20.193.80:3300/0 conn(0x7f8b800cde80 0x7f8b800c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient: _add_conns ranks=[1,2,5,4,0,3] 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5f881527f0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5f88073530 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f5f8815fad0 0x7f5f880ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient(hunting): picked mon.noname-c con 0x7f5f8815fad0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f880739a0 con 0x7f5f881527f0 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f8815d9d0 con 0x7f5f88073530 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5f88074b10 con 0x7f5f8815fad0 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f5f8e144640 10 monclient(hunting): authenticate will time out at 1082.348811s 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.621+0000 7f85effd3640 10 monclient: shutdown 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f85effd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f85effd3640 1 -- 10.20.193.6:0/3654791709 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e815da30 msgr2=0x7f85e815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f85effd3640 1 --2- 10.20.193.6:0/3654791709 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e815da30 0x7f85e815c880 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f85e815f380 tx=0x7f85dc02ee60 comp rx=0 tx=0).stop 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b854d2640 10 monclient(hunting): get_auth_request con 0x7f8b800cde80 auth_method 0 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b854d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b854d2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b854d2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7fe611ed3640 1 Processor -- start 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7fe611ed3640 1 -- start start 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7fe611ed3640 10 monclient: init 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f55e2698640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f55e2698640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f55e2698640 10 monclient: monmap: 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.622779+0000 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.622779+0000 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.628 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f85effd3640 1 -- 10.20.193.6:0/3654791709 shutdown_connections 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f85effd3640 1 --2- 10.20.193.6:0/3654791709 >> v2:10.20.193.40:3300/0 conn(0x7f85e80ccb40 0x7f85e80caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f85effd3640 1 --2- 10.20.193.6:0/3654791709 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e815da30 0x7f85e815c880 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f85effd3640 1 -- 10.20.193.6:0/3654791709 >> 10.20.193.6:0/3654791709 conn(0x7f85e80cc020 msgr2=0x7f85e80cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b84cd1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b84cd1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b84cd1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7fccddd74640 1 --2- 10.20.193.6:0/3661876388 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcce0074d60 0x7fcce00cc210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcce008f620 tx=0x7fccc802edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec99df5ba1ca27c7 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f55e2698640 1 Processor -- start 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b854d2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b854d2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f8b854d2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7fe611ed3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7fe611ed3640 10 monclient: _add_conns ranks=[3,4,0,2,5,1] 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7fe611ed3640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fe60c150770 0x7fe60c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 10 monclient(hunting): picked mon.noname-a con 0x7fe60c150770 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe60c15b210 0x7fe60c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 10 monclient(hunting): picked mon.noname-b con 0x7fe60c15b210 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe60c0c79e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe60c0c98f0 con 0x7fe60c0c79e0 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f55e2698640 1 -- start start 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.622+0000 7f55e2698640 10 monclient: init 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fe60c0cbff0 con 0x7fe60c150770 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe60c0c7550 con 0x7fe60c15b210 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe611ed3640 10 monclient(hunting): authenticate will time out at 1082.350254s 2026-02-22T23:23:27.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f85effd3640 1 -- 10.20.193.6:0/3654791709 shutdown_connections 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fe60c150770 0x7fe60c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f85effd3640 1 -- 10.20.193.6:0/3654791709 wait complete. 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient: _add_conns ranks=[2,5,0,4,1,3] 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f55dc150770 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f55dc15b210 0x7f55dc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient(hunting): picked mon.noname-c con 0x7f55dc15b210 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fe60c150770 0x7fe60c0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58656/0 (socket says 10.20.193.6:58656) 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3422766376 learned_addr learned my addr 10.20.193.6:0/3422766376 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 10 monclient(hunting): get_auth_request con 0x7fe60c150770 auth_method 0 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f55dc0cbab0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55dc0c98f0 con 0x7f55dc0cbab0 2026-02-22T23:23:27.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55dc0c6c20 con 0x7f55dc150770 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f55dc0c7550 con 0x7f55dc15b210 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f55e2698640 10 monclient(hunting): authenticate will time out at 1082.350662s 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f4465632640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f4465632640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f4465632640 10 monclient: monmap: 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.624345+0000 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.624345+0000 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 10 monclient(hunting): handle_auth_done global_id 4436 payload 274 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 1 -- 10.20.193.6:0/1079165846 >> v2:10.20.193.40:3300/0 conn(0x7f8b80155ef0 msgr2=0x7f8b8015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 1 --2- 10.20.193.6:0/1079165846 >> v2:10.20.193.40:3300/0 conn(0x7f8b80155ef0 0x7f8b8015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 1 -- 10.20.193.6:0/1079165846 >> v1:10.20.193.40:6789/0 conn(0x7f8b801562d0 legacy=0x7f8b800cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7f8b854d2640 1 -- 10.20.193.6:0/1079165846 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b800ce690 con 0x7f8b800cde80 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.623+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient: monmap: 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f4465632640 1 Processor -- start 2026-02-22T23:23:27.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 librados: starting msgr at 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 librados: starting objecter 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 Processor -- start 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 -- start start 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 librados: setting wanted keys 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 librados: calling monclient init 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient: init 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7fc27da43640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7fc27da43640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7fc27da43640 10 monclient: monmap: 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.625060+0000 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.625060+0000 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f5f86ffd640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f5f88073530 0x7f5f8815f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:48108/0 (socket says 10.20.193.6:48108) 2026-02-22T23:23:27.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f5f86ffd640 1 -- 10.20.193.6:0/1346291690 learned_addr learned my addr 10.20.193.6:0/1346291690 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7fc27da43640 1 Processor -- start 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e80ccb40 0x7f85e814f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): picked mon.noname-a con 0x7f85e80ccb40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e8155f80 0x7f85e8151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): picked mon.noname-b con 0x7f85e8155f80 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e8151670 0x7f85e8070320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): picked mon.noname-c con 0x7f85e8151670 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8072eb0 con 0x7f85e80ccb40 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85e806d240 con 0x7f85e8155f80 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8156360 con 0x7f85e8151670 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f85effd3640 10 monclient(hunting): authenticate will time out at 1082.351912s 2026-02-22T23:23:27.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7fc27da43640 1 -- start start 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.624+0000 7f4465632640 1 -- start start 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4465632640 10 monclient: init 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient: init 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_done global_id 4398 payload 274 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f8b854d2640 1 --2- 10.20.193.6:0/1079165846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b800cde80 0x7f8b800c9080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b800cd930 tx=0x7f8b7c02f250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffb4804747dd351 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3422766376 >> v2:10.20.193.40:3300/0 conn(0x7fe60c15b210 msgr2=0x7fe60c0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 1 --2- 10.20.193.6:0/3422766376 >> v2:10.20.193.40:3300/0 conn(0x7fe60c15b210 0x7fe60c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3422766376 >> v1:10.20.193.6:6789/0 conn(0x7fe60c0c79e0 legacy=0x7fe60c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3422766376 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c0c6c20 con 0x7fe60c150770 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f55e0c0e640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f55dc0cbab0 0x7f55dc15d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58766/0 (socket says 10.20.193.6:58766) 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f55e0c0e640 1 -- 10.20.193.6:0/2324883882 learned_addr learned my addr 10.20.193.6:0/2324883882 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4465632640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4d5308f640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4d5308f640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4d5308f640 10 monclient: monmap: 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.626409+0000 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.626409+0000 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4465632640 10 monclient: _add_conns ranks=[2,0,4,3,1,5] 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4465632640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f446015a770 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient: _add_conns ranks=[2,5,1,0,4,3] 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc27815f650 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc27815fa30 0x7fc2780d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient(hunting): picked mon.noname-c con 0x7fc27815fa30 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc2780d1ea0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780d2380 con 0x7fc2780d1ea0 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780d2660 con 0x7fc27815f650 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc2780c9550 con 0x7fc27815fa30 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7fc27da43640 10 monclient(hunting): authenticate will time out at 1082.352708s 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4465632640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4460160470 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.625+0000 7f4465632640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f44600ca160 0x7f44601608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4465632640 10 monclient(hunting): picked mon.noname-b con 0x7f44600ca160 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4465632640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4460160f60 con 0x7f4460160470 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4465632640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4460161240 con 0x7f446015a770 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4465632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4465632640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f446015ab50 con 0x7f44600ca160 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4465632640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4465632640 10 monclient(hunting): authenticate will time out at 1082.353120s 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7fd0a1b5c640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7fd0a1b5c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7fd0a1b5c640 10 monclient: monmap: 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.626883+0000 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.626883+0000 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.638 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 1 Processor -- start 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f887012c640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f887012c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f887012c640 10 monclient: monmap: 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.627268+0000 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.627268+0000 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.639 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7fd0a1b5c640 1 Processor -- start 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7fd0a1b5c640 1 -- start start 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7fd0a1b5c640 10 monclient: init 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 1 -- start start 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 10 monclient: init 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 10 monclient: _add_conns ranks=[4,3,5,0,1,2] 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4d4c1527f0 0x7f4d4c073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 10 monclient(hunting): picked mon.noname-b con 0x7f4d4c1527f0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f4d4c15d2c0 0x7f4d4c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 10 monclient(hunting): picked mon.noname-a con 0x7f4d4c15d2c0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.626+0000 7f4d5308f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4d4c15d730 0x7f4d4c15f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 10 monclient(hunting): picked mon.noname-c con 0x7f4d4c15d730 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4c074b10 con 0x7f4d4c15d2c0 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4c073c20 con 0x7f4d4c1527f0 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4c073f00 con 0x7f4d4c15d730 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f4d5308f640 10 monclient(hunting): authenticate will time out at 1082.354057s 2026-02-22T23:23:27.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ee549640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e8151670 0x7f85e8070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 1 Processor -- start 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ed547640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e8155f80 0x7f85e8151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient: _add_conns ranks=[3,5,1,0,2,4] 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd09c15f650 0x7fd09c15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient(hunting): picked mon.noname-a con 0x7fd09c15f650 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd09c0ceca0 0x7fd09c0d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient(hunting): picked mon.noname-c con 0x7fd09c0ceca0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd09c0d1ea0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd09c0d2450 con 0x7fd09c0d1ea0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd09c0c8dd0 con 0x7fd09c15f650 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd09c0d2730 con 0x7fd09c0ceca0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fd0a1b5c640 10 monclient(hunting): authenticate will time out at 1082.354452s 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7fe60b7fe640 1 --2- 10.20.193.6:0/3422766376 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c150770 0x7fe60c0cccb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe60c073180 tx=0x7fe5f802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685150fb43f38ccd server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ed547640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e8155f80 0x7f85e8151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33008/0 (socket says 10.20.193.6:33008) 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ed547640 1 -- 10.20.193.6:0/3222516727 learned_addr learned my addr 10.20.193.6:0/3222516727 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ed547640 10 monclient(hunting): get_auth_request con 0x7f85e8155f80 auth_method 0 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ed547640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ed547640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f85ed547640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 1 -- start start 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient: init 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient: _add_conns ranks=[2,0,4,1,5,3] 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f886815f650 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f886815fa30 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f88680c9b00 0x7f88680c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient(hunting): picked mon.noname-b con 0x7f88680c9b00 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88680ca4b0 con 0x7f886815fa30 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88680d2450 con 0x7f886815f650 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f88680c8dd0 con 0x7f88680c9b00 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.627+0000 7f887012c640 10 monclient(hunting): authenticate will time out at 1082.354804s 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f5f87fff640 1 --2- 10.20.193.6:0/1346291690 >> v2:10.20.193.80:3300/0 conn(0x7f5f8815fad0 0x7f5f880ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f85ee549640 10 monclient(hunting): get_auth_request con 0x7f85e8151670 auth_method 0 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f85ee549640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f85ee549640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f85ee549640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f65e6f11640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f65e6f11640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f65e6f11640 10 monclient: monmap: 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.629034+0000 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.629034+0000 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f85ed547640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f85ed547640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f85ed547640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f4d50e04640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4d4c1527f0 0x7f4d4c073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f4d50e04640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f4d4c1527f0 0x7f4d4c073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33026/0 (socket says 10.20.193.6:33026) 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f4d50e04640 1 -- 10.20.193.6:0/3656440994 learned_addr learned my addr 10.20.193.6:0/3656440994 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f4d50e04640 10 monclient(hunting): get_auth_request con 0x7f4d4c1527f0 auth_method 0 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f4d50e04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f4d50e04640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f4d50e04640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.628+0000 7f65e6f11640 1 Processor -- start 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 1 -- start start 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient: init 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f87fff640 10 monclient(hunting): get_auth_request con 0x7f5f8815fad0 auth_method 0 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f87fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f87fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f87fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f85ee549640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f85ee549640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f85ee549640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f867fc640 1 -- 10.20.193.6:0/1346291690 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3332448040 0 0) 0x7f5f8815d9d0 con 0x7f5f88073530 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f867fc640 10 monclient(hunting): _negotiate 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f867fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f867fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f867fc640 10 monclient(hunting): my global_id is 4439 2026-02-22T23:23:27.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f5f867fc640 1 -- 10.20.193.6:0/1346291690 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f6c003620 con 0x7f5f88073530 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7fc2767fc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc27815fa30 0x7fc2780d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient: _add_conns ranks=[2,4,5,0,1,3] 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f65e015f650 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f65e015fa30 0x7f65e00d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient(hunting): picked mon.noname-b con 0x7f65e015fa30 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f65e00d1ea0 0x7f65e00c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient(hunting): picked mon.noname-c con 0x7f65e00d1ea0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e00d2450 con 0x7f65e015f650 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f65e00c8dd0 con 0x7f65e015fa30 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f65e00d2730 con 0x7f65e00d1ea0 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.629+0000 7f65e6f11640 10 monclient(hunting): authenticate will time out at 1082.356870s 2026-02-22T23:23:27.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 10 monclient(hunting): handle_auth_done global_id 4423 payload 824 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 1 -- 10.20.193.6:0/3222516727 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e8151670 msgr2=0x7f85e8070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e8151670 0x7f85e8070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 1 -- 10.20.193.6:0/3222516727 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e80ccb40 msgr2=0x7f85e814f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e80ccb40 0x7f85e814f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f85ed547640 1 -- 10.20.193.6:0/3222516727 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e81f4750 con 0x7f85e8155f80 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7ff607ccd640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7ff607ccd640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7ff607ccd640 10 monclient: monmap: 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.630922+0000 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.630922+0000 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7ff607ccd640 1 Processor -- start 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7ff607ccd640 1 -- start start 2026-02-22T23:23:27.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7ff607ccd640 10 monclient: init 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f4d43fff640 1 --2- 10.20.193.6:0/3656440994 >> v2:10.20.193.6:3300/0 conn(0x7f4d4c15d2c0 0x7f4d4c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f4d50e04640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f4d50e04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.630+0000 7f4d50e04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc2767fc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc27815fa30 0x7fc2780d1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43326/0 (socket says 10.20.193.6:43326) 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc2767fc640 1 -- 10.20.193.6:0/1436554424 learned_addr learned my addr 10.20.193.6:0/1436554424 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc2767fc640 10 monclient(hunting): get_auth_request con 0x7fc27815fa30 auth_method 0 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc2767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc2767fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc2767fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f4d43fff640 10 monclient(hunting): get_auth_request con 0x7f4d4c15d2c0 auth_method 0 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f4d43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f4d43fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f4d43fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fd09b7fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd09c15f650 0x7fd09c15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient: _add_conns ranks=[3,1,2,4,5,0] 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff60015f650 0x7ff60015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient(hunting): picked mon.noname-a con 0x7ff60015f650 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff6000ceca0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff6000c9b00 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6000d2450 con 0x7ff6000ceca0 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f5f87fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f5f87fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6000d2730 con 0x7ff6000c9b00 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc37b4f5640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc37b4f5640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff6000c8dd0 con 0x7ff60015f650 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7fc37b4f5640 10 monclient: monmap: 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.632155+0000 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.632155+0000 2026-02-22T23:23:27.648 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f5f87fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7ff607ccd640 10 monclient(hunting): authenticate will time out at 1082.358612s 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f4d43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f4d43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.631+0000 7f4d43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f55daffd640 1 -- 10.20.193.6:0/2324883882 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1801591562 0 0) 0x7f55dc0c98f0 con 0x7f55dc0cbab0 2026-02-22T23:23:27.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 1 Processor -- start 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f55daffd640 10 monclient(hunting): _negotiate 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f55daffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f55daffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f55daffd640 10 monclient(hunting): my global_id is 4401 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f55daffd640 1 -- 10.20.193.6:0/2324883882 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55c0003620 con 0x7f55dc0cbab0 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc2767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc2767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc2767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 1 -- start start 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient: init 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fd09b7fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd09c15f650 0x7fd09c15fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58684/0 (socket says 10.20.193.6:58684) 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fd09b7fe640 1 -- 10.20.193.6:0/3389337341 learned_addr learned my addr 10.20.193.6:0/3389337341 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fd09b7fe640 10 monclient(hunting): get_auth_request con 0x7fd09c15f650 auth_method 0 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fd09b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fd09b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fd09b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 10 monclient(hunting): handle_auth_done global_id 4426 payload 274 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 1 -- 10.20.193.6:0/3656440994 >> v2:10.20.193.80:3300/0 conn(0x7f4d4c15d730 msgr2=0x7f4d4c15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 1 --2- 10.20.193.6:0/3656440994 >> v2:10.20.193.80:3300/0 conn(0x7f4d4c15d730 0x7f4d4c15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 1 -- 10.20.193.6:0/3656440994 >> v2:10.20.193.6:3300/0 conn(0x7f4d4c15d2c0 msgr2=0x7f4d4c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 1 --2- 10.20.193.6:0/3656440994 >> v2:10.20.193.6:3300/0 conn(0x7f4d4c15d2c0 0x7f4d4c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f4d50e04640 1 -- 10.20.193.6:0/3656440994 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c0741e0 con 0x7f4d4c1527f0 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f445e7fc640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f4460160470 0x7f44600c9a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58774/0 (socket says 10.20.193.6:58774) 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f445e7fc640 1 -- 10.20.193.6:0/3722843555 learned_addr learned my addr 10.20.193.6:0/3722843555 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient: _add_conns ranks=[2,4,5,3,0,1] 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc36c050bc0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc36c050fa0 0x7fc36c0e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient(hunting): picked mon.noname-b con 0x7fc36c050fa0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc36c0e0fa0 0x7fc36c0dfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient(hunting): picked mon.noname-c con 0x7fc36c0e0fa0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc36c04f430 con 0x7fc36c050bc0 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc36c04e0b0 con 0x7fc36c050fa0 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc36c04e390 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7fc37b4f5640 10 monclient(hunting): authenticate will time out at 1082.359727s 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f85ed547640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e8155f80 0x7f85e8151130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85dc02f410 tx=0x7f85dc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f85d67fc640 10 client.?.objecter ms_handle_connect 0x7f85e8155f80 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.632+0000 7f85d67fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 10 monclient(hunting): handle_auth_done global_id 4448 payload 274 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 1 -- 10.20.193.6:0/1346291690 >> v1:10.20.193.80:6789/0 conn(0x7f5f88073530 legacy=0x7f5f8815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 1 -- 10.20.193.6:0/1346291690 >> v1:10.20.193.40:6789/0 conn(0x7f5f881527f0 legacy=0x7f5f8815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7f5f87fff640 1 -- 10.20.193.6:0/1346291690 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f88073c20 con 0x7f5f8815fad0 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 1 -- 10.20.193.6:0/3661876388 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc803c070 con 0x7fcce0074d60 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: dump: 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 1 -- 10.20.193.6:0/3661876388 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc802fb00 con 0x7fcce0074d60 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 1 -- 10.20.193.6:0/3661876388 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8037440 con 0x7fcce0074d60 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 5 monclient: authenticate success, global_id 4414 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 4 set_mon_vals no callback set 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fccdd573640 10 monclient: dump: 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 10 monclient: shutdown 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 -- 10.20.193.6:0/3661876388 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcce0074d60 msgr2=0x7fcce00cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 --2- 10.20.193.6:0/3661876388 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcce0074d60 0x7fcce00cc210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcce008f620 tx=0x7fccc802edf0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 -- 10.20.193.6:0/3661876388 shutdown_connections 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 --2- 10.20.193.6:0/3661876388 >> v2:10.20.193.6:3300/0 conn(0x7fcce0156250 0x7fcce015d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 --2- 10.20.193.6:0/3661876388 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcce0074d60 0x7fcce00cc210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 -- 10.20.193.6:0/3661876388 >> 10.20.193.6:0/3661876388 conn(0x7fcce00ce700 msgr2=0x7fcce00ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 -- 10.20.193.6:0/3661876388 shutdown_connections 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.633+0000 7fcce4f1b640 1 -- 10.20.193.6:0/3661876388 wait complete. 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 10 monclient(hunting): handle_auth_done global_id 4451 payload 274 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 1 -- 10.20.193.6:0/1436554424 >> v1:10.20.193.80:6789/0 conn(0x7fc27815f650 legacy=0x7fc2780cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 1 -- 10.20.193.6:0/1436554424 >> v1:10.20.193.40:6789/0 conn(0x7fc2780d1ea0 legacy=0x7fc2780c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fc2767fc640 1 -- 10.20.193.6:0/1436554424 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2780c8f00 con 0x7fc27815fa30 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f4d50e04640 1 --2- 10.20.193.6:0/3656440994 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c073530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c06e9a0 tx=0x7f4d3c02ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6276879d1dd694d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fcce4f1b640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fcce4f1b640 10 monclient: monmap: 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fe0fda1d640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fe0fda1d640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fe0fda1d640 10 monclient: monmap: 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.635183+0000 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.635183+0000 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.655 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fcce4f1b640 1 librados: starting msgr at 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fcce4f1b640 1 librados: starting objecter 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 1 -- 10.20.193.6:0/1079165846 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c03c070 con 0x7f8b800cde80 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fcce4f1b640 1 Processor -- start 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: dump: 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.656 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 1 -- 10.20.193.6:0/1079165846 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7c037440 con 0x7f8b800cde80 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 1 -- 10.20.193.6:0/1079165846 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c037810 con 0x7f8b800cde80 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7fcce4f1b640 1 -- start start 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b777fe640 10 monclient: dump: 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b86f5c640 5 monclient: authenticate success, global_id 4436 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b86f5c640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 librados: setting wanted keys 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 librados: calling monclient init 2026-02-22T23:23:27.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient: init 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.634+0000 7f8b86f5c640 4 set_mon_vals no callback set 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fe0fda1d640 1 Processor -- start 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_done global_id 4407 payload 274 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 1 -- 10.20.193.6:0/3389337341 >> v2:10.20.193.80:3300/0 conn(0x7fd09c0ceca0 msgr2=0x7fd09c0d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 1 --2- 10.20.193.6:0/3389337341 >> v2:10.20.193.80:3300/0 conn(0x7fd09c0ceca0 0x7fd09c0d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 1 -- 10.20.193.6:0/3389337341 >> v1:10.20.193.40:6789/0 conn(0x7fd09c0d1ea0 legacy=0x7fd09c0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fe0fda1d640 1 -- start start 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fe0fda1d640 10 monclient: init 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.658 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fd09b7fe640 1 -- 10.20.193.6:0/3389337341 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c0c9020 con 0x7fd09c15f650 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7f8b86f5c640 10 monclient: shutdown 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7f8b86f5c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7f8b86f5c640 1 -- 10.20.193.6:0/1079165846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b800cde80 msgr2=0x7f8b800c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7f8b86f5c640 1 --2- 10.20.193.6:0/1079165846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b800cde80 0x7f8b800c9080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b800cd930 tx=0x7f8b7c02f250 comp rx=0 tx=0).stop 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcce0074d60 0x7fcce01f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): picked mon.noname-b con 0x7fcce0074d60 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcce0155e70 0x7fcce00c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): picked mon.noname-a con 0x7fcce0155e70 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcce0156250 0x7fcce0142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): picked mon.noname-c con 0x7fcce0156250 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00cc750 con 0x7fcce0155e70 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00cd9b0 con 0x7fcce0074d60 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00cc8d0 con 0x7fcce0156250 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fcce4f1b640 10 monclient(hunting): authenticate will time out at 1082.362667s 2026-02-22T23:23:27.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7f5f87fff640 1 --2- 10.20.193.6:0/1346291690 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 0x7f5f880ccf70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009d90 tx=0x7f5f7c02f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34410e36e514f9cc server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fe0fda1d640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fe0fda1d640 10 monclient: _add_conns ranks=[0,4,2,3,1,5] 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.635+0000 7fe0fda1d640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fe0f815f650 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe0f815fa30 0x7fe0f80d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 10 monclient(hunting): picked mon.noname-b con 0x7fe0f815fa30 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 1 -- 10.20.193.6:0/1079165846 shutdown_connections 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe0f80d1ea0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 1 --2- 10.20.193.6:0/1079165846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b800cde80 0x7f8b800c9080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 1 --2- 10.20.193.6:0/1079165846 >> v2:10.20.193.40:3300/0 conn(0x7f8b80155ef0 0x7f8b8015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f80d2450 con 0x7fe0f815f650 2026-02-22T23:23:27.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fc2767fc640 1 --2- 10.20.193.6:0/1436554424 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2780d1960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc260009970 tx=0x7fc26002ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3393e5a99746d7b3 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 1 -- 10.20.193.6:0/1079165846 >> 10.20.193.6:0/1079165846 conn(0x7f8b8006d8c0 msgr2=0x7f8b8006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f80d2730 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f80c8dd0 con 0x7fe0f815fa30 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe0fda1d640 10 monclient(hunting): authenticate will time out at 1082.363074s 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 1 -- 10.20.193.6:0/1079165846 shutdown_connections 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f445f7fe640 1 --2- 10.20.193.6:0/3722843555 >> v2:10.20.193.40:3300/0 conn(0x7f44600ca160 0x7f44601608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 1 -- 10.20.193.6:0/1079165846 wait complete. 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f55db7fe640 1 --2- 10.20.193.6:0/2324883882 >> v2:10.20.193.80:3300/0 conn(0x7f55dc15b210 0x7f55dc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7f8b86f5c640 10 monclient: monmap: 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.661 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7ff677598640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7ff677598640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7ff677598640 10 monclient: monmap: 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.637429+0000 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.637429+0000 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 1 -- 10.20.193.6:0/3422766376 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f803c070 con 0x7fe60c150770 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: dump: 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 1 -- 10.20.193.6:0/3422766376 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f802fb40 con 0x7fe60c150770 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe611ed3640 5 monclient: authenticate success, global_id 4398 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 librados: starting msgr at 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe611ed3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe611ed3640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 1 -- 10.20.193.6:0/3422766376 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8037440 con 0x7fe60c150770 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe611ed3640 4 set_mon_vals no callback set 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 librados: starting objecter 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.636+0000 7fe60a7fc640 10 monclient: dump: 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 Processor -- start 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f55db7fe640 10 monclient(hunting): get_auth_request con 0x7f55dc15b210 auth_method 0 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f55db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f55db7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f55db7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 -- start start 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 librados: setting wanted keys 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 librados: calling monclient init 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 10 monclient: init 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7ff677598640 1 Processor -- start 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fd09b7fe640 1 --2- 10.20.193.6:0/3389337341 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c15f650 0x7fd09c15fa30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ce0a0 tx=0x7fd08802ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da61da1cb2e953e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7ff677598640 1 -- start start 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7ff677598640 10 monclient: init 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 10 monclient: shutdown 2026-02-22T23:23:27.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 -- 10.20.193.6:0/3422766376 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c150770 msgr2=0x7fe60c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 --2- 10.20.193.6:0/3422766376 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c150770 0x7fe60c0cccb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe60c073180 tx=0x7fe5f802eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 -- 10.20.193.6:0/3422766376 shutdown_connections 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 --2- 10.20.193.6:0/3422766376 >> v2:10.20.193.40:3300/0 conn(0x7fe60c15b210 0x7fe60c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 --2- 10.20.193.6:0/3422766376 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c150770 0x7fe60c0cccb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 -- 10.20.193.6:0/3422766376 >> 10.20.193.6:0/3422766376 conn(0x7fe60c0cc190 msgr2=0x7fe60c0cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 -- 10.20.193.6:0/3422766376 shutdown_connections 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7fe611ed3640 1 -- 10.20.193.6:0/3422766376 wait complete. 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f445f7fe640 10 monclient(hunting): get_auth_request con 0x7f44600ca160 auth_method 0 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f445f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f445f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f445f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f886d6a0640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f886815fa30 0x7f88680d1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58790/0 (socket says 10.20.193.6:58790) 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f886d6a0640 1 -- 10.20.193.6:0/59849103 learned_addr learned my addr 10.20.193.6:0/59849103 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f65d7fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f65e015fa30 0x7f65e00d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8b800cde80 0x7f8b801f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 10 monclient(hunting): picked mon.noname-a con 0x7f8b800cde80 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.665 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.637+0000 7f8b86f5c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b80155ef0 0x7f8b801f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient: monmap: 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 10 monclient(hunting): picked mon.noname-c con 0x7f8b80155ef0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8b801562d0 0x7f8b800d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 10 monclient(hunting): picked mon.noname-b con 0x7f8b801562d0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient: _add_conns ranks=[3,0,1,4,2,5] 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8b800cef30 con 0x7f8b800cde80 2026-02-22T23:23:27.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f445f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f445f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff6701527f0 0x7ff670073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8b800cec50 con 0x7f8b801562d0 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f445f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient(hunting): picked mon.noname-a con 0x7ff6701527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8b800c95c0 con 0x7f8b80155ef0 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff67015d2c0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7f8b86f5c640 10 monclient(hunting): authenticate will time out at 1082.365166s 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff67015d730 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff670073c20 con 0x7ff67015d2c0 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff670073f00 con 0x7ff67015d730 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 librados: starting msgr at 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 librados: starting objecter 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff670074b10 con 0x7ff6701527f0 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7ff677598640 10 monclient(hunting): authenticate will time out at 1082.365290s 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 Processor -- start 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 -- start start 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 librados: setting wanted keys 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 librados: calling monclient init 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient: init 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c0c79e0 0x7fe60c156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): picked mon.noname-a con 0x7fe60c0c79e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe60c150770 0x7fe60c1f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): picked mon.noname-c con 0x7fe60c150770 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe60c15b210 0x7fe60c1f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): picked mon.noname-b con 0x7fe60c15b210 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c150b50 con 0x7fe60c0c79e0 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0c71e0 con 0x7fe60c15b210 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0c9500 con 0x7fe60c150770 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.638+0000 7fe611ed3640 10 monclient(hunting): authenticate will time out at 1082.365921s 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f65d7fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f65e015fa30 0x7f65e00d1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33040/0 (socket says 10.20.193.6:33040) 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f65d7fff640 1 -- 10.20.193.6:0/1445637866 learned_addr learned my addr 10.20.193.6:0/1445637866 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f65d7fff640 10 monclient(hunting): get_auth_request con 0x7f65e015fa30 auth_method 0 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f65d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f65d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f65d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55daffd640 1 -- 10.20.193.6:0/2324883882 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138110229 0 0) 0x7f55dc0c6c20 con 0x7f55dc150770 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55daffd640 10 monclient(hunting): _negotiate 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55daffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55daffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55daffd640 10 monclient(hunting): my global_id is 4442 2026-02-22T23:23:27.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55daffd640 1 -- 10.20.193.6:0/2324883882 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55dc0c98f0 con 0x7f55dc150770 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7ff605a42640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff60015f650 0x7ff60015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.639+0000 7f55db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f886e6a2640 1 --2- 10.20.193.6:0/59849103 >> v2:10.20.193.40:3300/0 conn(0x7f88680c9b00 0x7f88680c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 10 monclient(hunting): handle_auth_done global_id 4429 payload 274 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 1 -- 10.20.193.6:0/3722843555 >> v1:10.20.193.80:6789/0 conn(0x7f446015a770 legacy=0x7f4460164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 1 -- 10.20.193.6:0/3722843555 >> v1:10.20.193.6:6789/0 conn(0x7f4460160470 legacy=0x7f44600c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f445f7fe640 1 -- 10.20.193.6:0/3722843555 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44600c8f70 con 0x7f44600ca160 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f65d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f65d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.640+0000 7f65d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f886e6a2640 10 monclient(hunting): get_auth_request con 0x7f88680c9b00 auth_method 0 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f886e6a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f886e6a2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f886e6a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 10 monclient(hunting): handle_auth_done global_id 4454 payload 274 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 1 -- 10.20.193.6:0/2324883882 >> v1:10.20.193.80:6789/0 conn(0x7f55dc150770 legacy=0x7f55dc0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 1 -- 10.20.193.6:0/2324883882 >> v1:10.20.193.6:6789/0 conn(0x7f55dc0cbab0 legacy=0x7f55dc15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7ff605a42640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff60015f650 0x7ff60015fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58694/0 (socket says 10.20.193.6:58694) 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7ff605a42640 1 -- 10.20.193.6:0/2408565424 learned_addr learned my addr 10.20.193.6:0/2408565424 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7ff605a42640 10 monclient(hunting): get_auth_request con 0x7ff60015f650 auth_method 0 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7ff605a42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7ff605a42640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7ff605a42640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f55db7fe640 1 -- 10.20.193.6:0/2324883882 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc0c6f00 con 0x7f55dc15b210 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f886e6a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f886e6a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.641+0000 7f886e6a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7fccddd74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcce0155e70 0x7fcce00c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7f445f7fe640 1 --2- 10.20.193.6:0/3722843555 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f44600ca160 0x7f44601608b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4450009d90 tx=0x7f445002f390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868aebe47ae0731a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7ff605a42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7ff605a42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7ff605a42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7fccddd74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcce0155e70 0x7fcce00c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58696/0 (socket says 10.20.193.6:58696) 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7fccddd74640 1 -- 10.20.193.6:0/4269421126 learned_addr learned my addr 10.20.193.6:0/4269421126 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7fccddd74640 10 monclient(hunting): get_auth_request con 0x7fcce0155e70 auth_method 0 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7fccddd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7fccddd74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.642+0000 7fccddd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 10 monclient(hunting): handle_auth_done global_id 4435 payload 274 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 1 -- 10.20.193.6:0/59849103 >> v1:10.20.193.80:6789/0 conn(0x7f886815f650 legacy=0x7f88680ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 1 -- 10.20.193.6:0/59849103 >> v1:10.20.193.6:6789/0 conn(0x7f886815fa30 legacy=0x7f88680d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.643+0000 7f886e6a2640 1 -- 10.20.193.6:0/59849103 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88680d2730 con 0x7f88680c9b00 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.644+0000 7fccddd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.644+0000 7fccddd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.644+0000 7fccddd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.645+0000 7f55db7fe640 1 --2- 10.20.193.6:0/2324883882 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 0x7f55dc0cccb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0098a0 tx=0x7f55cc02ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e688857effca5251 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.645+0000 7f886e6a2640 1 --2- 10.20.193.6:0/59849103 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88680c9b00 0x7f88680c9f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8864009d90 tx=0x7f886402f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=334d76777807f5c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 10 monclient(hunting): handle_auth_done global_id 4419 payload 824 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 1 -- 10.20.193.6:0/4269421126 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcce0156250 msgr2=0x7fcce0142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcce0156250 0x7fcce0142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 1 -- 10.20.193.6:0/4269421126 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcce0074d60 msgr2=0x7fcce01f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcce0074d60 0x7fcce01f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7fccddd74640 1 -- 10.20.193.6:0/4269421126 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce01f8580 con 0x7fcce0155e70 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 10 monclient(hunting): handle_auth_done global_id 4416 payload 274 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 1 -- 10.20.193.6:0/2408565424 >> v1:10.20.193.80:6789/0 conn(0x7ff6000c9b00 legacy=0x7ff6000c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 1 -- 10.20.193.6:0/2408565424 >> v1:10.20.193.40:6789/0 conn(0x7ff6000ceca0 legacy=0x7ff6000d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7ff605a42640 1 -- 10.20.193.6:0/2408565424 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6000c9020 con 0x7ff60015f650 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 1 -- 10.20.193.6:0/3656440994 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c03c070 con 0x7f4d4c1527f0 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: dump: 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 1 -- 10.20.193.6:0/3656440994 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c02fb50 con 0x7f4d4c1527f0 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 1 -- 10.20.193.6:0/3656440994 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c037440 con 0x7f4d4c1527f0 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d5308f640 5 monclient: authenticate success, global_id 4426 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d5308f640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d5308f640 4 set_mon_vals no callback set 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.646+0000 7f4d437fe640 10 monclient: dump: 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.674 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 10 monclient: shutdown 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 -- 10.20.193.6:0/3656440994 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4d4c1527f0 msgr2=0x7f4d4c073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 --2- 10.20.193.6:0/3656440994 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c073530 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c06e9a0 tx=0x7f4d3c02ee90 comp rx=0 tx=0).stop 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 -- 10.20.193.6:0/3656440994 shutdown_connections 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 --2- 10.20.193.6:0/3656440994 >> v2:10.20.193.80:3300/0 conn(0x7f4d4c15d730 0x7f4d4c15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 --2- 10.20.193.6:0/3656440994 >> v2:10.20.193.6:3300/0 conn(0x7f4d4c15d2c0 0x7f4d4c15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 --2- 10.20.193.6:0/3656440994 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c073530 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 -- 10.20.193.6:0/3656440994 >> 10.20.193.6:0/3656440994 conn(0x7f4d4c0ce730 msgr2=0x7f4d4c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7ff67530d640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff6701527f0 0x7ff670073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 -- 10.20.193.6:0/3656440994 shutdown_connections 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.647+0000 7f4d5308f640 1 -- 10.20.193.6:0/3656440994 wait complete. 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc03c040 con 0x7f85e8155f80 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: dump: 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.675 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85dc042630 con 0x7f85e8155f80 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc042a20 con 0x7f85e8155f80 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 5 monclient: authenticate success, global_id 4423 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d67fc640 10 monclient: dump: 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.676 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 monclient: _renew_subs 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85d7fff640 4 set_mon_vals no callback set 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e806f560 con 0x7f85e8155f80 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 client.4423.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 monclient: _renew_subs 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e806fb20 con 0x7f85e8155f80 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 1 librados: init done 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 1 -- 10.20.193.6:0/1436554424 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26003c070 con 0x7fc27815fa30 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: dump: 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.677 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 1 -- 10.20.193.6:0/1436554424 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc26002fac0 con 0x7fc27815fa30 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 1 -- 10.20.193.6:0/1436554424 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260037440 con 0x7fc27815fa30 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 5 monclient: authenticate success, global_id 4451 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 4 set_mon_vals no callback set 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc275ffb640 10 monclient: dump: 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fccddd74640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcce0155e70 0x7fcce00c6c70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccc802f3a0 tx=0x7fccc80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fccc6ffd640 10 client.?.objecter ms_handle_connect 0x7fcce0155e70 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fccc6ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 10 monclient: monmap: 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 1 librados: starting msgr at 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 1 librados: starting objecter 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 1 Processor -- start 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 1 -- start start 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 1 librados: setting wanted keys 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 1 librados: calling monclient init 2026-02-22T23:23:27.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f4d5308f640 10 monclient: init 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 client.4423.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 15 client.4423.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 20 client.4423.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 client.4423.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85e8076e00 con 0x7f85e8155f80 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 10 monclient: shutdown 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 1 -- 10.20.193.6:0/1436554424 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 msgr2=0x7fc2780d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 1 --2- 10.20.193.6:0/1436554424 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2780d1960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc260009970 tx=0x7fc26002ee90 comp rx=0 tx=0).stop 2026-02-22T23:23:27.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7ff605a42640 1 --2- 10.20.193.6:0/2408565424 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff60015f650 0x7ff60015fa30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009970 tx=0x7ff5f002ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dbd9a6e9ef1c4f server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 1 -- 10.20.193.6:0/1436554424 shutdown_connections 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 1 --2- 10.20.193.6:0/1436554424 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2780d1960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 1 -- 10.20.193.6:0/1436554424 >> 10.20.193.6:0/1436554424 conn(0x7fc278072fc0 msgr2=0x7fc2780733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 1 -- 10.20.193.6:0/1436554424 shutdown_connections 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.648+0000 7fc27da43640 1 -- 10.20.193.6:0/1436554424 wait complete. 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 10 monclient(hunting): handle_auth_done global_id 4432 payload 274 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 1 -- 10.20.193.6:0/1445637866 >> v2:10.20.193.80:3300/0 conn(0x7f65e00d1ea0 msgr2=0x7f65e00c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 1 --2- 10.20.193.6:0/1445637866 >> v2:10.20.193.80:3300/0 conn(0x7f65e00d1ea0 0x7f65e00c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 1 -- 10.20.193.6:0/1445637866 >> v1:10.20.193.80:6789/0 conn(0x7f65e015f650 legacy=0x7f65e00ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f65d7fff640 1 -- 10.20.193.6:0/1445637866 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e00c9020 con 0x7f65e015fa30 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c0c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): picked mon.noname-c con 0x7f4d4c1527f0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4d4c15d2c0 0x7f4d4c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): picked mon.noname-b con 0x7f4d4c15d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4d4c15d730 0x7f4d4c1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): picked mon.noname-a con 0x7f4d4c15d730 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c074fa0 con 0x7f4d4c15d730 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c0747a0 con 0x7f4d4c15d2c0 2026-02-22T23:23:27.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c075120 con 0x7f4d4c1527f0 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7ff67530d640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff6701527f0 0x7ff670073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58714/0 (socket says 10.20.193.6:58714) 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7ff67530d640 1 -- 10.20.193.6:0/1689781373 learned_addr learned my addr 10.20.193.6:0/1689781373 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7f4d5308f640 10 monclient(hunting): authenticate will time out at 1082.376244s 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7ff67530d640 10 monclient(hunting): get_auth_request con 0x7ff6701527f0 auth_method 0 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7ff67530d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7ff67530d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7ff67530d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7fc27da43640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7fc27da43640 10 monclient: monmap: 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.683 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7fc379cf2640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc36c050fa0 0x7fc36c0e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7fc27da43640 1 librados: starting msgr at 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7fc27da43640 1 librados: starting objecter 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.649+0000 7fc27da43640 1 Processor -- start 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 -- start start 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 librados: setting wanted keys 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 librados: calling monclient init 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient: init 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7ff67530d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7ff67530d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7ff67530d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fe60b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c0c79e0 0x7fe60c156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2780d1ea0 0x7fc27814c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): picked mon.noname-a con 0x7fc2780d1ea0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc27815f650 0x7fc2781566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): picked mon.noname-b con 0x7fc27815f650 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2781f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): picked mon.noname-c con 0x7fc27815fa30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc278160260 con 0x7fc2780d1ea0 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2781600e0 con 0x7fc27815f650 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780ca0f0 con 0x7fc27815fa30 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.650+0000 7fc27da43640 10 monclient(hunting): authenticate will time out at 1082.377901s 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.651+0000 7fc37acf4640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc36c0e0fa0 0x7fc36c0dfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.651+0000 7fc37acf4640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc36c0e0fa0 0x7fc36c0dfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43360/0 (socket says 10.20.193.6:43360) 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.651+0000 7fc37acf4640 1 -- 10.20.193.6:0/1107268186 learned_addr learned my addr 10.20.193.6:0/1107268186 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.651+0000 7fc37acf4640 10 monclient(hunting): get_auth_request con 0x7fc36c0e0fa0 auth_method 0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.651+0000 7fc37acf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.651+0000 7fc37acf4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.651+0000 7fc37acf4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.652+0000 7fc37acf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.652+0000 7fc37acf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.652+0000 7fc37acf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 10 monclient(hunting): handle_auth_done global_id 4460 payload 274 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 1 -- 10.20.193.6:0/1107268186 >> v2:10.20.193.40:3300/0 conn(0x7fc36c050fa0 msgr2=0x7fc36c0e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 1 --2- 10.20.193.6:0/1107268186 >> v2:10.20.193.40:3300/0 conn(0x7fc36c050fa0 0x7fc36c0e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 1 -- 10.20.193.6:0/1107268186 >> v1:10.20.193.80:6789/0 conn(0x7fc36c050bc0 legacy=0x7fc36c04d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fc37acf4640 1 -- 10.20.193.6:0/1107268186 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c04e670 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 10 monclient(hunting): handle_auth_done global_id 4422 payload 274 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 1 -- 10.20.193.6:0/1689781373 >> v1:10.20.193.40:6789/0 conn(0x7ff67015d730 legacy=0x7ff6700ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 1 -- 10.20.193.6:0/1689781373 >> v1:10.20.193.6:6789/0 conn(0x7ff67015d2c0 legacy=0x7ff67015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7ff67530d640 1 -- 10.20.193.6:0/1689781373 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6700741e0 con 0x7ff6701527f0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fe60b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c0c79e0 0x7fe60c156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58720/0 (socket says 10.20.193.6:58720) 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3918289228 learned_addr learned my addr 10.20.193.6:0/3918289228 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fe60b7fe640 10 monclient(hunting): get_auth_request con 0x7fe60c0c79e0 auth_method 0 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fe60b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fe60b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7fe60b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7f8b77fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b80155ef0 0x7f8b801f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.653+0000 7f8b84cd1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8b800cde80 0x7f8b801f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b80155ef0 0x7f8b801f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43364/0 (socket says 10.20.193.6:43364) 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7fe0f67fc640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe0f815fa30 0x7fe0f80d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 1 -- 10.20.193.6:0/3080835411 learned_addr learned my addr 10.20.193.6:0/3080835411 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 10 monclient(hunting): get_auth_request con 0x7f8b80155ef0 auth_method 0 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7fc37acf4640 1 --2- 10.20.193.6:0/1107268186 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c0dfa00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc36c04d4f0 tx=0x7fc37002f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4e06591bcf07a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b84cd1640 10 monclient(hunting): get_auth_request con 0x7f8b800cde80 auth_method 0 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b84cd1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b84cd1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b84cd1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.654+0000 7f8b77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 1 -- 10.20.193.6:0/1346291690 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c03c070 con 0x7f5f8815fad0 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: dump: 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 1 -- 10.20.193.6:0/1346291690 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7c037440 con 0x7f5f8815fad0 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 1 -- 10.20.193.6:0/1346291690 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c037810 con 0x7f5f8815fad0 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f867fc640 10 monclient: dump: 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.688 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f8e144640 5 monclient: authenticate success, global_id 4448 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f8e144640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f8e144640 4 set_mon_vals no callback set 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe0f67fc640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe0f815fa30 0x7fe0f80d1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33086/0 (socket says 10.20.193.6:33086) 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe0f67fc640 1 -- 10.20.193.6:0/2479573250 learned_addr learned my addr 10.20.193.6:0/2479573250 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe0f67fc640 10 monclient(hunting): get_auth_request con 0x7fe0f815fa30 auth_method 0 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe0f67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe0f67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe0f67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f8b854d2640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8b801562d0 0x7f8b800d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 10 monclient(hunting): handle_auth_done global_id 4425 payload 824 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3918289228 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe60c150770 msgr2=0x7fe60c1f07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe60c150770 0x7fe60c1f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3918289228 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe60c15b210 msgr2=0x7fe60c1f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe60c15b210 0x7fe60c1f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7fe60b7fe640 1 -- 10.20.193.6:0/3918289228 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c1efd00 con 0x7fe60c0c79e0 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f8e144640 10 monclient: shutdown 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f8e144640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f8e144640 1 -- 10.20.193.6:0/1346291690 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 msgr2=0x7f5f880ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.655+0000 7f5f8e144640 1 --2- 10.20.193.6:0/1346291690 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 0x7f5f880ccf70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009d90 tx=0x7f5f7c02f1e0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 1 -- 10.20.193.6:0/1346291690 shutdown_connections 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 1 --2- 10.20.193.6:0/1346291690 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 0x7f5f880ccf70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 1 -- 10.20.193.6:0/1346291690 >> 10.20.193.6:0/1346291690 conn(0x7f5f880ce730 msgr2=0x7f5f880ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 1 -- 10.20.193.6:0/1346291690 shutdown_connections 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 1 -- 10.20.193.6:0/1346291690 wait complete. 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 1 -- 10.20.193.6:0/3722843555 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445003c070 con 0x7f44600ca160 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: dump: 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.690 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 1 -- 10.20.193.6:0/3722843555 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4450037640 con 0x7f44600ca160 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 5 monclient: authenticate success, global_id 4429 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 1 -- 10.20.193.6:0/3722843555 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450037a50 con 0x7f44600ca160 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 4 set_mon_vals no callback set 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f445dffb640 10 monclient: dump: 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b854d2640 10 monclient(hunting): get_auth_request con 0x7f8b801562d0 auth_method 0 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b854d2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b854d2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b854d2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 10 monclient: monmap: 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 10 monclient: shutdown 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 1 -- 10.20.193.6:0/3722843555 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f44600ca160 msgr2=0x7f44601608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f4465632640 1 --2- 10.20.193.6:0/3722843555 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f44600ca160 0x7f44601608b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4450009d90 tx=0x7f445002f390 comp rx=0 tx=0).stop 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 10 monclient(hunting): handle_auth_done global_id 4463 payload 824 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 1 -- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8b801562d0 msgr2=0x7f8b800d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8b801562d0 0x7f8b800d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 1 -- 10.20.193.6:0/3080835411 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8b800cde80 msgr2=0x7f8b801f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8b800cde80 0x7f8b801f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f8b77fff640 1 -- 10.20.193.6:0/3080835411 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b801f4a30 con 0x7f8b80155ef0 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 1 librados: starting msgr at 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.656+0000 7f5f8e144640 1 librados: starting objecter 2026-02-22T23:23:27.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 Processor -- start 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 -- start start 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 librados: setting wanted keys 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f4465632640 1 -- 10.20.193.6:0/3722843555 shutdown_connections 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f4465632640 1 --2- 10.20.193.6:0/3722843555 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f44600ca160 0x7f44601608b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 librados: calling monclient init 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient: init 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f4465632640 1 -- 10.20.193.6:0/3722843555 >> 10.20.193.6:0/3722843555 conn(0x7f446006d8e0 msgr2=0x7f446006dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f4465632640 1 -- 10.20.193.6:0/3722843555 shutdown_connections 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f4465632640 1 -- 10.20.193.6:0/3722843555 wait complete. 2026-02-22T23:23:27.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7ff67530d640 1 --2- 10.20.193.6:0/1689781373 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff6701527f0 0x7ff670073530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff660009970 tx=0x7ff66002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419991028b997c1c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5f88073530 0x7f5f881f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): picked mon.noname-b con 0x7f5f88073530 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5f881527f0 0x7f5f881f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): picked mon.noname-a con 0x7f5f881527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 0x7f5f8814d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): picked mon.noname-c con 0x7f5f8815fad0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880744c0 con 0x7f5f881527f0 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880741e0 con 0x7f5f88073530 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5f880747a0 con 0x7f5f8815fad0 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f5f8e144640 10 monclient(hunting): authenticate will time out at 1082.384748s 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f4465632640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.657+0000 7f4465632640 10 monclient: monmap: 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.694 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 librados: starting msgr at 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 librados: starting objecter 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 Processor -- start 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 -- start start 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 librados: setting wanted keys 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 librados: calling monclient init 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient: init 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7fe5e7fff640 10 client.?.objecter ms_handle_connect 0x7fe60c0c79e0 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7fe60b7fe640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c0c79e0 0x7fe60c156480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8002410 tx=0x7fe5f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7fe5e7fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f44600ca160 0x7f4460153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): picked mon.noname-a con 0x7f44600ca160 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f446015a770 0x7f4460151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): picked mon.noname-b con 0x7f446015a770 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4460160470 0x7f446014c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): picked mon.noname-c con 0x7f4460160470 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f44600c9530 con 0x7f44600ca160 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f44600c8d20 con 0x7f446015a770 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f446006e690 con 0x7f4460160470 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.658+0000 7f4465632640 10 monclient(hunting): authenticate will time out at 1082.385956s 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 1 -- 10.20.193.6:0/3389337341 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08803c070 con 0x7fd09c15f650 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: dump: 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 1 -- 10.20.193.6:0/3389337341 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08802fa90 con 0x7fd09c15f650 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 1 -- 10.20.193.6:0/3389337341 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08802fe60 con 0x7fd09c15f650 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 5 monclient: authenticate success, global_id 4407 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 4 set_mon_vals no callback set 2026-02-22T23:23:27.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd09a7fc640 10 monclient: dump: 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 1 -- 10.20.193.6:0/2324883882 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc03c070 con 0x7f55dc15b210 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: dump: 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 1 -- 10.20.193.6:0/2324883882 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55cc02fb70 con 0x7f55dc15b210 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 1 -- 10.20.193.6:0/2324883882 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc037440 con 0x7f55dc15b210 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55daffd640 10 monclient: dump: 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.697 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55e2698640 5 monclient: authenticate success, global_id 4454 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55e2698640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55e2698640 4 set_mon_vals no callback set 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 10 monclient: shutdown 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/3389337341 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c15f650 msgr2=0x7fd09c15fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 --2- 10.20.193.6:0/3389337341 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c15f650 0x7fd09c15fa30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ce0a0 tx=0x7fd08802ee60 comp rx=0 tx=0).stop 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/3389337341 shutdown_connections 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 --2- 10.20.193.6:0/3389337341 >> v2:10.20.193.80:3300/0 conn(0x7fd09c0ceca0 0x7fd09c0d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 --2- 10.20.193.6:0/3389337341 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c15f650 0x7fd09c15fa30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/3389337341 >> 10.20.193.6:0/3389337341 conn(0x7fd09c072fc0 msgr2=0x7fd09c0733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/3389337341 shutdown_connections 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/3389337341 wait complete. 2026-02-22T23:23:27.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f4d50e04640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c0c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55e2698640 10 monclient: shutdown 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55e2698640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55e2698640 1 -- 10.20.193.6:0/2324883882 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 msgr2=0x7f55dc0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.659+0000 7f55e2698640 1 --2- 10.20.193.6:0/2324883882 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 0x7f55dc0cccb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0098a0 tx=0x7f55cc02ee60 comp rx=0 tx=0).stop 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 -- 10.20.193.6:0/2324883882 shutdown_connections 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 --2- 10.20.193.6:0/2324883882 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 0x7f55dc0cccb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 -- 10.20.193.6:0/2324883882 >> 10.20.193.6:0/2324883882 conn(0x7f55dc0cc190 msgr2=0x7f55dc0cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 -- 10.20.193.6:0/2324883882 shutdown_connections 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 -- 10.20.193.6:0/2324883882 wait complete. 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 10 monclient: monmap: 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f4d50e04640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c0c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43372/0 (socket says 10.20.193.6:43372) 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f4d50e04640 1 -- 10.20.193.6:0/1771385440 learned_addr learned my addr 10.20.193.6:0/1771385440 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f4d50e04640 10 monclient(hunting): get_auth_request con 0x7f4d4c1527f0 auth_method 0 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f4d50e04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f4d50e04640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f4d50e04640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 1 librados: starting msgr at 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 1 librados: starting objecter 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 1 Processor -- start 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 1 -- start start 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 1 librados: setting wanted keys 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 1 librados: calling monclient init 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7fd0a1b5c640 10 monclient: init 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 10 monclient: monmap: 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.700 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 librados: starting msgr at 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 librados: starting objecter 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 Processor -- start 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 -- start start 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 librados: setting wanted keys 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 1 librados: calling monclient init 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.660+0000 7f55e2698640 10 monclient: init 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c0ceca0 0x7fd09c1f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f4d50e04640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f4d50e04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f4d50e04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): picked mon.noname-a con 0x7fd09c0ceca0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd09c0d1ea0 0x7fd09c1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): picked mon.noname-b con 0x7fd09c0d1ea0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd09c15f650 0x7fd09c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): picked mon.noname-c con 0x7fd09c15f650 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c0cf0f0 con 0x7fd09c0ceca0 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c0c95e0 con 0x7fd09c0d1ea0 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1f46b0 con 0x7fd09c15f650 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fd0a1b5c640 10 monclient(hunting): authenticate will time out at 1082.388254s 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7fc2777fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2781f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f55dc0cbab0 0x7f55dc1e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): picked mon.noname-b con 0x7f55dc0cbab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f55dc150770 0x7f55dc1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): picked mon.noname-a con 0x7f55dc150770 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 0x7f55dc158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): picked mon.noname-c con 0x7f55dc15b210 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc0cbec0 con 0x7f55dc150770 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc150b50 con 0x7f55dc0cbab0 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc0c9500 con 0x7f55dc15b210 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.661+0000 7f55e2698640 10 monclient(hunting): authenticate will time out at 1082.388620s 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 10 monclient(hunting): handle_auth_done global_id 4466 payload 824 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 1 -- 10.20.193.6:0/1771385440 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4d4c15d2c0 msgr2=0x7f4d4c157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4d4c15d2c0 0x7f4d4c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 1 -- 10.20.193.6:0/1771385440 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4d4c15d730 msgr2=0x7f4d4c1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4d4c15d730 0x7f4d4c1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7f4d50e04640 1 -- 10.20.193.6:0/1771385440 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c0c6750 con 0x7f4d4c1527f0 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7fc2777fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2781f90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43382/0 (socket says 10.20.193.6:43382) 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7fc2777fe640 1 -- 10.20.193.6:0/268222076 learned_addr learned my addr 10.20.193.6:0/268222076 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7fc2777fe640 10 monclient(hunting): get_auth_request con 0x7fc27815fa30 auth_method 0 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7fc2777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7fc2777fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.662+0000 7fc2777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 1 -- 10.20.193.6:0/2408565424 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f003c070 con 0x7ff60015f650 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: dump: 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.703 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 1 -- 10.20.193.6:0/2408565424 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f002fac0 con 0x7ff60015f650 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 1 -- 10.20.193.6:0/2408565424 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0037440 con 0x7ff60015f650 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff607ccd640 5 monclient: authenticate success, global_id 4416 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff607ccd640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff607ccd640 4 set_mon_vals no callback set 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff604a40640 10 monclient: dump: 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.663+0000 7ff607ccd640 10 monclient: shutdown 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 -- 10.20.193.6:0/2408565424 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff60015f650 msgr2=0x7ff60015fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 --2- 10.20.193.6:0/2408565424 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff60015f650 0x7ff60015fa30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009970 tx=0x7ff5f002ee90 comp rx=0 tx=0).stop 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 -- 10.20.193.6:0/2408565424 shutdown_connections 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 --2- 10.20.193.6:0/2408565424 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff60015f650 0x7ff60015fa30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 -- 10.20.193.6:0/2408565424 >> 10.20.193.6:0/2408565424 conn(0x7ff600072fc0 msgr2=0x7ff6000733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 -- 10.20.193.6:0/2408565424 shutdown_connections 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 -- 10.20.193.6:0/2408565424 wait complete. 2026-02-22T23:23:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7f8b77fff640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b80155ef0 0x7f8b801f3b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c480 tx=0x7f8b6800c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7f8b757fa640 10 client.?.objecter ms_handle_connect 0x7f8b80155ef0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7f8b757fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 1 -- 10.20.193.6:0/1107268186 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37002fcd0 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: dump: 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 1 -- 10.20.193.6:0/1107268186 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3700373f0 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 1 -- 10.20.193.6:0/1107268186 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3700377c0 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc3794f1640 10 monclient: dump: 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc37b4f5640 5 monclient: authenticate success, global_id 4460 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc37b4f5640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc37b4f5640 4 set_mon_vals no callback set 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f85dc02fce0 con 0x7f85e8155f80 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 10 monclient: monmap: 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 librados: starting msgr at 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 librados: starting objecter 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 Processor -- start 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 -- start start 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 librados: setting wanted keys 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 1 librados: calling monclient init 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7ff607ccd640 10 monclient: init 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc37b4f5640 10 monclient: shutdown 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc37b4f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc37b4f5640 1 -- 10.20.193.6:0/1107268186 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 msgr2=0x7fc36c0dfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.664+0000 7fc37b4f5640 1 --2- 10.20.193.6:0/1107268186 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c0dfa00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc36c04d4f0 tx=0x7fc37002f270 comp rx=0 tx=0).stop 2026-02-22T23:23:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7f4d50e04640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c0c9000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c002410 tx=0x7f4d3c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7f4d417fa640 10 client.?.objecter ms_handle_connect 0x7f4d4c1527f0 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7f4d417fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 -- 10.20.193.6:0/1107268186 shutdown_connections 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 --2- 10.20.193.6:0/1107268186 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c0dfa00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 --2- 10.20.193.6:0/1107268186 >> v2:10.20.193.40:3300/0 conn(0x7fc36c050fa0 0x7fc36c0e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 -- 10.20.193.6:0/1107268186 >> 10.20.193.6:0/1107268186 conn(0x7fc36c006e10 msgr2=0x7fc36c0071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 -- 10.20.193.6:0/1107268186 shutdown_connections 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 -- 10.20.193.6:0/1107268186 wait complete. 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7f85d67fc640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b8078e00 0x7f85b807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff6000c9b00 0x7ff6001f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): picked mon.noname-b con 0x7ff6000c9b00 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6000ceca0 0x7ff60015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): picked mon.noname-c con 0x7ff6000ceca0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff60015f650 0x7ff60015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): picked mon.noname-a con 0x7ff60015f650 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000cf0f0 con 0x7ff60015f650 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7f85edd48640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b8078e00 0x7f85b807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7f85edd48640 10 monclient: get_auth_request con 0x7f85b8078e00 auth_method 0 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000c95e0 con 0x7ff6000c9b00 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff60014fb80 con 0x7ff6000ceca0 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7ff607ccd640 10 monclient(hunting): authenticate will time out at 1082.392594s 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 10 monclient: monmap: 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.708 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7f445effd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f44600ca160 0x7f4460153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 librados: starting msgr at 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 librados: starting objecter 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.665+0000 7fc37b4f5640 1 Processor -- start 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f85edd48640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b8078e00 0x7f85b807b2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85d8002770 tx=0x7f85d8007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f85d67fc640 10 client.4423.objecter ms_handle_connect 0x7f85b8078e00 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 -- start start 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 librados: setting wanted keys 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 librados: calling monclient init 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient: init 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f445effd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f44600ca160 0x7f4460153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58760/0 (socket says 10.20.193.6:58760) 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f445effd640 1 -- 10.20.193.6:0/1220287504 learned_addr learned my addr 10.20.193.6:0/1220287504 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f445effd640 10 monclient(hunting): get_auth_request con 0x7f44600ca160 auth_method 0 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f445effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f445effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7f445effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc36c050bc0 0x7fc36c0d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): picked mon.noname-b con 0x7fc36c050bc0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc36c050fa0 0x7fc36c0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): picked mon.noname-a con 0x7fc36c050fa0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c00f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): picked mon.noname-c con 0x7fc36c0e0fa0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c04ef10 con 0x7fc36c050fa0 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c04ec30 con 0x7fc36c050bc0 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc36c0dff40 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc37b4f5640 10 monclient(hunting): authenticate will time out at 1082.393560s 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.666+0000 7fc2777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f445effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f445effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f445effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7fc276ffd640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2780d1ea0 0x7fc27814c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f85dc02fab0 con 0x7f85e8155f80 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 10 client.4423.objecter ms_dispatch2 0x7f85e8000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 3 client.4423.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 3 client.4423.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 10 client.4423.objecter checking command 1 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7f85b8081340 0x7f85b8083780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 20 client.4423.objecter _get_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 20 client.4423.objecter put_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 15 client.4423.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 20 client.4423.objecter _get_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 20 client.4423.objecter get_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 15 client.4423.objecter _session_command_op_assign 1 1 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 20 client.4423.objecter put_session s=0x7f85b8080c70 osd=1 4 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 10 client.4423.objecter _send_command 1 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 --> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85dc045ec0 con 0x7f85b8081340 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 20 client.4423.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f85dc0838c0 con 0x7f85e8155f80 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85d67fc640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.667+0000 7f85ee549640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7f85b8081340 0x7f85b8083780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f85ee549640 10 monclient: get_auth_request con 0x7f85b8081340 auth_method 0 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fe0f67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fe0f67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fe0f67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 10 monclient(hunting): handle_auth_done global_id 4434 payload 824 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 1 -- 10.20.193.6:0/1220287504 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4460160470 msgr2=0x7f446014c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4460160470 0x7f446014c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 1 -- 10.20.193.6:0/1220287504 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f446015a770 msgr2=0x7f4460151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fc276ffd640 10 monclient(hunting): get_auth_request con 0x7fc2780d1ea0 auth_method 0 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fc276ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fc276ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f446015a770 0x7f4460151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fc276ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7f445effd640 1 -- 10.20.193.6:0/1220287504 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44601f8d00 con 0x7f44600ca160 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8045020 con 0x7fcce0155e70 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: dump: 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc803d040 con 0x7fcce0155e70 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc803c070 con 0x7fcce0155e70 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 5 monclient: authenticate success, global_id 4419 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 10 monclient: _renew_subs 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccdcd72640 4 set_mon_vals no callback set 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce01f4f90 con 0x7fcce0155e70 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fccc6ffd640 10 monclient: dump: 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.713 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 10 client.4419.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 10 monclient: _renew_subs 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce00c5eb0 con 0x7fcce0155e70 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.668+0000 7fcce4f1b640 1 librados: init done 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fcce4f1b640 10 client.4419.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fcce4f1b640 15 client.4419.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fcce4f1b640 20 client.4419.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fcce4f1b640 10 client.4419.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fcce4f1b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcce00cc210 con 0x7fcce0155e70 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 10 monclient(hunting): handle_auth_done global_id 4469 payload 824 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f5f86ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5f881527f0 0x7f5f881f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 1 -- 10.20.193.6:0/268222076 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc27815f650 msgr2=0x7fc2781566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc27815f650 0x7fc2781566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 1 -- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2780d1ea0 msgr2=0x7fc27814c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2780d1ea0 0x7fc27814c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7fc2777fe640 1 -- 10.20.193.6:0/268222076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27814aa00 con 0x7fc27815fa30 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 1 -- 10.20.193.6:0/59849103 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886403c070 con 0x7f88680c9b00 2026-02-22T23:23:27.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: dump: 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 1 -- 10.20.193.6:0/59849103 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886402fe10 con 0x7f88680c9b00 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 1 -- 10.20.193.6:0/59849103 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864037660 con 0x7f88680c9b00 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f886ce9f640 10 monclient: dump: 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f887012c640 5 monclient: authenticate success, global_id 4435 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f887012c640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.669+0000 7f887012c640 4 set_mon_vals no callback set 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5f881527f0 0x7f5f881f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58752/0 (socket says 10.20.193.6:58752) 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 1 -- 10.20.193.6:0/1363421420 learned_addr learned my addr 10.20.193.6:0/1363421420 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 10 monclient(hunting): get_auth_request con 0x7f5f881527f0 auth_method 0 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 10 monclient: shutdown 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f85ee549640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7f85b8081340 0x7f85b8083780 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 1 -- 10.20.193.6:0/59849103 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88680c9b00 msgr2=0x7f88680c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 1 --2- 10.20.193.6:0/59849103 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88680c9b00 0x7f88680c9f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8864009d90 tx=0x7f886402f150 comp rx=0 tx=0).stop 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f85d67fc640 10 client.4423.objecter ms_handle_connect 0x7f85b8081340 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 10 monclient(hunting): handle_auth_done global_id 4441 payload 274 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 1 -- 10.20.193.6:0/59849103 shutdown_connections 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 1 --2- 10.20.193.6:0/59849103 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88680c9b00 0x7f88680c9f70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 1 -- 10.20.193.6:0/2479573250 >> v1:10.20.193.80:6789/0 conn(0x7fe0f80d1ea0 legacy=0x7fe0f80c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f445effd640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f44600ca160 0x7f4460153b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f445400c700 tx=0x7f445400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f443f7fe640 10 client.?.objecter ms_handle_connect 0x7f44600ca160 2026-02-22T23:23:27.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f443f7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 1 -- 10.20.193.6:0/59849103 >> 10.20.193.6:0/59849103 conn(0x7f8868072fc0 msgr2=0x7f88680733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 1 -- 10.20.193.6:0/59849103 shutdown_connections 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 1 -- 10.20.193.6:0/2479573250 >> v1:10.20.193.6:6789/0 conn(0x7fe0f815f650 legacy=0x7fe0f80ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 1 -- 10.20.193.6:0/59849103 wait complete. 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7fe0f67fc640 1 -- 10.20.193.6:0/2479573250 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f80c9020 con 0x7fe0f815fa30 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f5f86ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.670+0000 7f887012c640 10 monclient: monmap: 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 librados: starting msgr at 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 librados: starting objecter 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 Processor -- start 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 -- start start 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 librados: setting wanted keys 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 librados: calling monclient init 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient: init 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7fc2777fe640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2781f90a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc26800b970 tx=0x7fc26800be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7fc24ffff640 10 client.?.objecter ms_handle_connect 0x7fc27815fa30 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7fc24ffff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88680c9b00 0x7f88681f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): picked mon.noname-a con 0x7f88680c9b00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f886815f650 0x7f8868156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): picked mon.noname-b con 0x7f886815f650 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f886815fa30 0x7f886814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): picked mon.noname-c con 0x7f886815fa30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f88680c95e0 con 0x7f88680c9b00 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88680c9300 con 0x7f886815f650 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f88680d2070 con 0x7f886815fa30 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.671+0000 7f887012c640 10 monclient(hunting): authenticate will time out at 1082.398800s 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f87fff640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 0x7f5f8814d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 10 monclient(hunting): handle_auth_done global_id 4440 payload 824 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 1 -- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 msgr2=0x7f5f8814d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5f8815fad0 0x7f5f8814d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 1 -- 10.20.193.6:0/1363421420 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5f88073530 msgr2=0x7f5f881f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5f88073530 0x7f5f881f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7f5f86ffd640 1 -- 10.20.193.6:0/1363421420 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f880c6610 con 0x7f5f881527f0 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 1 -- 10.20.193.6:0/1689781373 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66003c070 con 0x7ff6701527f0 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: dump: 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 1 -- 10.20.193.6:0/1689781373 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66002fb60 con 0x7ff6701527f0 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 1 -- 10.20.193.6:0/1689781373 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff660037440 con 0x7ff6701527f0 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff65ffff640 10 monclient: dump: 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff677598640 5 monclient: authenticate success, global_id 4422 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff677598640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7ff677598640 4 set_mon_vals no callback set 2026-02-22T23:23:27.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.672+0000 7fd09b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c0ceca0 0x7fd09c1f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7fe0f67fc640 1 --2- 10.20.193.6:0/2479573250 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f815fa30 0x7fe0f80d1960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009970 tx=0x7fe0e002ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=facdb89c44f34623 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 10 monclient: shutdown 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 1 -- 10.20.193.6:0/1689781373 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff6701527f0 msgr2=0x7ff670073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 1 --2- 10.20.193.6:0/1689781373 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff6701527f0 0x7ff670073530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff660009970 tx=0x7ff66002eef0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 1 -- 10.20.193.6:0/1689781373 shutdown_connections 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 1 --2- 10.20.193.6:0/1689781373 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff6701527f0 0x7ff670073530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 1 -- 10.20.193.6:0/1689781373 >> 10.20.193.6:0/1689781373 conn(0x7ff6700ce730 msgr2=0x7ff6700ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 1 -- 10.20.193.6:0/1689781373 shutdown_connections 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7ff677598640 1 -- 10.20.193.6:0/1689781373 wait complete. 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7fd09b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c0ceca0 0x7fd09c1f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58776/0 (socket says 10.20.193.6:58776) 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.673+0000 7fd09b7fe640 1 -- 10.20.193.6:0/2662627114 learned_addr learned my addr 10.20.193.6:0/2662627114 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7fd09b7fe640 10 monclient(hunting): get_auth_request con 0x7fd09c0ceca0 auth_method 0 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7fd09b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7fd09b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7fd09b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 10 monclient: monmap: 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 librados: starting msgr at 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 librados: starting objecter 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 Processor -- start 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 -- start start 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 librados: setting wanted keys 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 librados: calling monclient init 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 10 monclient: init 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6701527f0 0x7ff6701f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 10 monclient(hunting): picked mon.noname-c con 0x7ff6701527f0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff67015d2c0 0x7ff67014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 10 monclient(hunting): picked mon.noname-b con 0x7ff67015d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.674+0000 7ff677598640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff67015d730 0x7ff6700c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 10 monclient(hunting): picked mon.noname-a con 0x7ff67015d730 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff670074fa0 con 0x7ff67015d730 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700747a0 con 0x7ff67015d2c0 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff670075120 con 0x7ff6701527f0 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7ff677598640 10 monclient(hunting): authenticate will time out at 1082.402047s 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7f65d7fff640 1 --2- 10.20.193.6:0/1445637866 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015fa30 0x7f65e00d1960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65e008fa30 tx=0x7f65c802ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6734558a1ce809ff server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.675+0000 7f55dbfff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f55dc0cbab0 0x7f55dc1e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68017070 con 0x7f8b80155ef0 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: dump: 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b680052f0 con 0x7f8b80155ef0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b680036f0 con 0x7f8b80155ef0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b757fa640 10 monclient: dump: 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 5 monclient: authenticate success, global_id 4463 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b76ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 monclient: _renew_subs 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8014cf80 con 0x7f8b80155ef0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 client.4463.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 monclient: _renew_subs 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b801f4750 con 0x7f8b80155ef0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 1 librados: init done 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fd09bfff640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd09c15f650 0x7fd09c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268002c70 con 0x7fc27815fa30 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: dump: 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.724 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 client.4463.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 15 client.4463.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 20 client.4463.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f5f86ffd640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5f881527f0 0x7f5f881f5a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c730 tx=0x7f5f7800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f5f63fff640 10 client.?.objecter ms_handle_connect 0x7f5f881527f0 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f5f63fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc268010070 con 0x7fc27815fa30 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 5 monclient: authenticate success, global_id 4469 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 10 monclient: _renew_subs 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 client.4463.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27814ace0 con 0x7fc27815fa30 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b80077010 con 0x7f8b80155ef0 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26800c9d0 con 0x7fc27815fa30 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc24ffff640 10 monclient: dump: 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc2757fa640 4 set_mon_vals no callback set 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 10 client.4469.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 10 monclient: _renew_subs 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27814b2a0 con 0x7fc27815fa30 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fc27da43640 1 librados: init done 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fd09bfff640 10 monclient(hunting): get_auth_request con 0x7fd09c15f650 auth_method 0 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fd09bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fd09bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.676+0000 7fd09bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44540052d0 con 0x7f44600ca160 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: dump: 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4454005470 con 0x7f44600ca160 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44540038d0 con 0x7f44600ca160 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f443f7fe640 10 monclient: dump: 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f445d7fa640 4 set_mon_vals no callback set 2026-02-22T23:23:27.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 5 monclient: authenticate success, global_id 4434 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 monclient: _renew_subs 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44600cf830 con 0x7f44600ca160 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fc27da43640 10 client.4469.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fc27da43640 15 client.4469.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fc27da43640 20 client.4469.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fc27da43640 10 client.4469.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fc27da43640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc27814a7b0 con 0x7fc27815fa30 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 client.4434.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 monclient: _renew_subs 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44600cfdf0 con 0x7f44600ca160 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 1 librados: init done 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8044070 con 0x7fe60c0c79e0 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: dump: 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.728 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f8040630 con 0x7fe60c0c79e0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 5 monclient: authenticate success, global_id 4425 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 10 monclient: _renew_subs 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c1f3740 con 0x7fe60c0c79e0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f803c040 con 0x7fe60c0c79e0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe5e7fff640 10 monclient: dump: 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe609ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 10 client.4425.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 10 monclient: _renew_subs 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c1f3cf0 con 0x7fe60c0c79e0 2026-02-22T23:23:27.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fe611ed3640 1 librados: init done 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c044070 con 0x7f4d4c1527f0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: dump: 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c040410 con 0x7f4d4c1527f0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c03c040 con 0x7f4d4c1527f0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d417fa640 10 monclient: dump: 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d42ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 5 monclient: authenticate success, global_id 4466 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 10 monclient: _renew_subs 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c156760 con 0x7f4d4c1527f0 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 10 client.4466.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 10 monclient: _renew_subs 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fd09bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fd09bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7fd09bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c156d20 con 0x7f4d4c1527f0 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4d5308f640 1 librados: init done 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 client.4434.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 15 client.4434.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 20 client.4434.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 client.4434.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.677+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44601f8fe0 con 0x7f44600ca160 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7fe611ed3640 10 client.4425.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7fe611ed3640 15 client.4425.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7fe611ed3640 20 client.4425.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7fe611ed3640 10 client.4425.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7fe611ed3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe60c0c7df0 con 0x7fe60c0c79e0 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f55e0c0e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 0x7f55dc158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f4d5308f640 10 client.4466.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f4d5308f640 15 client.4466.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f4d5308f640 20 client.4466.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f4d5308f640 10 client.4466.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f4d5308f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d4c152bd0 con 0x7f4d4c1527f0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f55e0c0e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 0x7f55dc158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43412/0 (socket says 10.20.193.6:43412) 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f55e0c0e640 1 -- 10.20.193.6:0/3130673925 learned_addr learned my addr 10.20.193.6:0/3130673925 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f55e0c0e640 10 monclient(hunting): get_auth_request con 0x7f55dc15b210 auth_method 0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f55e0c0e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f55e0c0e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.678+0000 7f55e0c0e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 10 monclient(hunting): handle_auth_done global_id 4443 payload 824 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 1 -- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd09c15f650 msgr2=0x7fd09c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd09c15f650 0x7fd09c15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 1 -- 10.20.193.6:0/2662627114 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd09c0d1ea0 msgr2=0x7fd09c1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd09c0d1ea0 0x7fd09c1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7fd09b7fe640 1 -- 10.20.193.6:0/2662627114 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c1f5080 con 0x7fd09c0ceca0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.679+0000 7f55db7fe640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f55dc150770 0x7f55dc1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.680+0000 7fd09b7fe640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c0ceca0 0x7fd09c1f76a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd088002410 tx=0x7fd0880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.680+0000 7fd077fff640 10 client.?.objecter ms_handle_connect 0x7fd09c0ceca0 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.680+0000 7fd077fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.682+0000 7f55dbfff640 10 monclient(hunting): get_auth_request con 0x7f55dc0cbab0 auth_method 0 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.682+0000 7f55dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.682+0000 7f55dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.682+0000 7f55dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.683+0000 7f55dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.683+0000 7f55dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.683+0000 7f55dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.685+0000 7f55db7fe640 10 monclient(hunting): get_auth_request con 0x7f55dc150770 auth_method 0 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.685+0000 7f55db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.685+0000 7f55db7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.685+0000 7f55db7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.685+0000 7f55db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.685+0000 7f55db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.685+0000 7f55db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 10 monclient(hunting): handle_auth_done global_id 4444 payload 824 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 1 -- 10.20.193.6:0/3130673925 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 msgr2=0x7f55dc158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f55dc15b210 0x7f55dc158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 1 -- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f55dc150770 msgr2=0x7f55dc1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f55dc150770 0x7f55dc1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.686+0000 7f55dbfff640 1 -- 10.20.193.6:0/3130673925 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc14d210 con 0x7f55dc0cbab0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.688+0000 7fc37acf4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c00f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.688+0000 7f886d6a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f886815f650 0x7f8868156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.688+0000 7f55dbfff640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f55dc0cbab0 0x7f55dc1e6620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c430 tx=0x7f55c800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.688+0000 7f55d8ff9640 10 client.?.objecter ms_handle_connect 0x7f55dc0cbab0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.688+0000 7f55d8ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7f886d6a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f886815f650 0x7f8868156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33204/0 (socket says 10.20.193.6:33204) 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7f886d6a0640 1 -- 10.20.193.6:0/2575574876 learned_addr learned my addr 10.20.193.6:0/2575574876 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7f886d6a0640 10 monclient(hunting): get_auth_request con 0x7f886815f650 auth_method 0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7f886d6a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7f886d6a0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7f886d6a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7fc37acf4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c00f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43418/0 (socket says 10.20.193.6:43418) 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7fc37acf4640 1 -- 10.20.193.6:0/996581608 learned_addr learned my addr 10.20.193.6:0/996581608 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7fc37acf4640 10 monclient(hunting): get_auth_request con 0x7fc36c0e0fa0 auth_method 0 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7fc37acf4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7fc37acf4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.689+0000 7fc37acf4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7fc37acf4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7fc37acf4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7fc37acf4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7ff605241640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6000ceca0 0x7ff60015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== osd.1 v2:10.20.193.40:6824/2813004313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f85dc045ec0 con 0x7f85b8081340 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f85d67fc640 10 client.4423.objecter ms_dispatch2 0x7f85e8000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f85d67fc640 10 client.4423.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f85d67fc640 20 client.4423.objecter put_session s=0x7f85b8080c70 osd=1 4 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f85d67fc640 15 client.4423.objecter _session_command_op_remove 1 1 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f886d6a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f886d6a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.690+0000 7f886d6a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7ff605a42640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff6000c9b00 0x7ff6001f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 10 monclient(hunting): handle_auth_done global_id 4478 payload 824 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37a4f3640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc36c050bc0 0x7fc36c0d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 1 -- 10.20.193.6:0/996581608 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc36c050bc0 msgr2=0x7fc36c0d28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc36c050bc0 0x7fc36c0d28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 1 -- 10.20.193.6:0/996581608 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc36c050fa0 msgr2=0x7fc36c0c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc36c050fa0 0x7fc36c0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7fc37acf4640 1 -- 10.20.193.6:0/996581608 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36c0d2e20 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7ff605a42640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff6000c9b00 0x7ff6001f94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33182/0 (socket says 10.20.193.6:33182) 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7ff605a42640 1 -- 10.20.193.6:0/1494014871 learned_addr learned my addr 10.20.193.6:0/1494014871 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7ff605a42640 10 monclient(hunting): get_auth_request con 0x7ff6000c9b00 auth_method 0 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7ff605a42640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7ff605a42640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.691+0000 7ff605a42640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7ff605241640 10 monclient(hunting): get_auth_request con 0x7ff6000ceca0 auth_method 0 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7ff605241640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7ff605241640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7ff605241640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7ff605a42640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7ff605a42640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7ff605a42640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 1 -- 10.20.193.6:0/2479573250 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e003c070 con 0x7fe0f815fa30 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: dump: 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.736 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 1 -- 10.20.193.6:0/2479573250 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e002fb10 con 0x7fe0f815fa30 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 1 -- 10.20.193.6:0/2479573250 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0037440 con 0x7fe0f815fa30 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0fda1d640 5 monclient: authenticate success, global_id 4441 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0fda1d640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0fda1d640 4 set_mon_vals no callback set 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7fe0f5ffb640 10 monclient: dump: 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.692+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4454003a70 con 0x7f44600ca160 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 10 monclient: shutdown 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 -- 10.20.193.6:0/2479573250 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f815fa30 msgr2=0x7fe0f80d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 --2- 10.20.193.6:0/2479573250 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f815fa30 0x7fe0f80d1960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009970 tx=0x7fe0e002ee90 comp rx=0 tx=0).stop 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 -- 10.20.193.6:0/2479573250 shutdown_connections 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 --2- 10.20.193.6:0/2479573250 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f815fa30 0x7fe0f80d1960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 -- 10.20.193.6:0/2479573250 >> 10.20.193.6:0/2479573250 conn(0x7fe0f8072fc0 msgr2=0x7fe0f80733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 -- 10.20.193.6:0/2479573250 shutdown_connections 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 -- 10.20.193.6:0/2479573250 wait complete. 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7f443f7fe640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f442c078e50 0x7f442c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7ff605241640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7ff605241640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7ff605241640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7f445e7fc640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f442c078e50 0x7f442c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7f445e7fc640 10 monclient: get_auth_request con 0x7f442c078e50 auth_method 0 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fc37acf4640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c00f740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc370009f50 tx=0x7fc370004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fc35affd640 10 client.?.objecter ms_handle_connect 0x7fc36c0e0fa0 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fc35affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 10 monclient: monmap: 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.738 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7f443f7fe640 10 client.4434.objecter ms_handle_connect 0x7f442c078e50 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7f445e7fc640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f442c078e50 0x7f442c07b310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4448007fd0 tx=0x7f4448007b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 librados: starting msgr at 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.693+0000 7fe0fda1d640 1 librados: starting objecter 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 10 monclient(hunting): handle_auth_done global_id 4450 payload 824 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 1 -- 10.20.193.6:0/1494014871 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6000ceca0 msgr2=0x7ff60015e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6000ceca0 0x7ff60015e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 1 -- 10.20.193.6:0/1494014871 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff60015f650 msgr2=0x7ff60015ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff60015f650 0x7ff60015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7ff605a42640 1 -- 10.20.193.6:0/1494014871 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60014ec90 con 0x7ff6000c9b00 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 Processor -- start 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 -- start start 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 librados: setting wanted keys 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 librados: calling monclient init 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient: init 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f80d1ea0 0x7fe0f80d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): picked mon.noname-b con 0x7fe0f80d1ea0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0f815f650 0x7fe0f814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): picked mon.noname-a con 0x7fe0f815f650 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe0f815fa30 0x7fe0f814cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): picked mon.noname-c con 0x7fe0f815fa30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f8160150 con 0x7fe0f815f650 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f80c95e0 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f81602d0 con 0x7fe0f815fa30 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.694+0000 7fe0fda1d640 10 monclient(hunting): authenticate will time out at 1082.421653s 2026-02-22T23:23:27.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7ff67530d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6701527f0 0x7ff6701f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 1 -- 10.20.193.6:0/1445637866 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c803c070 con 0x7f65e015fa30 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: dump: 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 1 -- 10.20.193.6:0/1445637866 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c802fa90 con 0x7f65e015fa30 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 1 -- 10.20.193.6:0/1445637866 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c802fe60 con 0x7f65e015fa30 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65d77fe640 10 monclient: dump: 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65e6f11640 5 monclient: authenticate success, global_id 4432 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65e6f11640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.695+0000 7f65e6f11640 4 set_mon_vals no callback set 2026-02-22T23:23:27.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f445405e130 con 0x7f44600ca160 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 10 client.4434.objecter ms_dispatch2 0x7f4460000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 3 client.4434.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 3 client.4434.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7ff605a42640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff6000c9b00 0x7ff6001f94b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5f003a040 tx=0x7ff5f00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7ff5ee7fc640 10 client.?.objecter ms_handle_connect 0x7ff6000c9b00 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7ff5ee7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 10 client.4434.objecter checking command 1 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7f442c081390 0x7f442c0837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 20 client.4434.objecter _get_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 20 client.4434.objecter put_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 15 client.4434.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 20 client.4434.objecter _get_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 20 client.4434.objecter get_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 15 client.4434.objecter _session_command_op_assign 14 1 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 20 client.4434.objecter put_session s=0x7f442c080cc0 osd=14 4 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 10 client.4434.objecter _send_command 1 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 --> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f442c083ea0 con 0x7f442c081390 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 20 client.4434.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f445405e4f0 con 0x7f44600ca160 2026-02-22T23:23:27.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f443f7fe640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 10 monclient: shutdown 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 -- 10.20.193.6:0/1445637866 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015fa30 msgr2=0x7f65e00d1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 --2- 10.20.193.6:0/1445637866 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015fa30 0x7f65e00d1960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65e008fa30 tx=0x7f65c802ee60 comp rx=0 tx=0).stop 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 -- 10.20.193.6:0/1445637866 shutdown_connections 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 --2- 10.20.193.6:0/1445637866 >> v2:10.20.193.80:3300/0 conn(0x7f65e00d1ea0 0x7f65e00c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 --2- 10.20.193.6:0/1445637866 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015fa30 0x7f65e00d1960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 -- 10.20.193.6:0/1445637866 >> 10.20.193.6:0/1445637866 conn(0x7f65e0072fc0 msgr2=0x7f65e00733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 -- 10.20.193.6:0/1445637866 shutdown_connections 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f65e6f11640 1 -- 10.20.193.6:0/1445637866 wait complete. 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f445f7fe640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7f442c081390 0x7f442c0837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.696+0000 7f445f7fe640 10 monclient: get_auth_request con 0x7f442c081390 auth_method 0 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient: monmap: 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.743 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 librados: starting msgr at 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 librados: starting objecter 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 Processor -- start 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 -- start start 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 librados: setting wanted keys 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 librados: calling monclient init 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient: init 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 10 client.4423.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 15 client.4423.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 20 client.4423.objecter _get_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 20 client.4423.objecter put_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 20 client.4423.objecter _get_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 15 client.4423.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 20 client.4423.objecter get_session s=0x7f85b8080c70 osd=1 3 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 15 client.4423.objecter _session_command_op_assign 1 2 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 20 client.4423.objecter put_session s=0x7f85b8080c70 osd=1 4 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 10 client.4423.objecter _send_command 2 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 --> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85e806f000 con 0x7f85b8081340 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 10 monclient(hunting): handle_auth_done global_id 4447 payload 824 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 1 -- 10.20.193.6:0/2575574876 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f886815fa30 msgr2=0x7f886814cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f886815fa30 0x7f886814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 1 -- 10.20.193.6:0/2575574876 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88680c9b00 msgr2=0x7f88681f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88680c9b00 0x7f88681f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f886d6a0640 1 -- 10.20.193.6:0/2575574876 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8868159a50 con 0x7f886815f650 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65e00d1ea0 0x7f65e014c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): picked mon.noname-a con 0x7f65e00d1ea0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015f650 0x7f65e00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): picked mon.noname-b con 0x7f65e015f650 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65e015fa30 0x7f65e0149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): picked mon.noname-c con 0x7f65e015fa30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0160150 con 0x7f65e00d1ea0 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65e00c95e0 con 0x7f65e015f650 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01602d0 con 0x7f65e015fa30 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.697+0000 7f65e6f11640 10 monclient(hunting): authenticate will time out at 1082.424853s 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85d67fc640 1 -- 10.20.193.6:0/3222516727 <== osd.1 v2:10.20.193.40:6824/2813004313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f85e806f000 con 0x7f85b8081340 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85d67fc640 10 client.4423.objecter ms_dispatch2 0x7f85e8000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85d67fc640 10 client.4423.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85d67fc640 20 client.4423.objecter put_session s=0x7f85b8080c70 osd=1 4 2026-02-22T23:23:27.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85d67fc640 15 client.4423.objecter _session_command_op_remove 1 2 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7ff67530d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6701527f0 0x7ff6701f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43440/0 (socket says 10.20.193.6:43440) 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7ff67530d640 1 -- 10.20.193.6:0/1016129297 learned_addr learned my addr 10.20.193.6:0/1016129297 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7ff67530d640 10 monclient(hunting): get_auth_request con 0x7ff6701527f0 auth_method 0 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7ff67530d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7ff67530d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7ff67530d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 10 librados: watch_flush enter 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 10 librados: watch_flush exit 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 10 client.4423.objecter close_session for osd.1 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7f85b8081340 msgr2=0x7f85b8083780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7f85b8081340 0x7f85b8083780 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 20 client.4423.objecter put_session s=0x7f85b8080c70 osd=1 1 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 20 client.4423.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 10 client.4423.objecter successfully canceled tick 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b8078e00 msgr2=0x7f85b807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b8078e00 0x7f85b807b2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85d8002770 tx=0x7f85d8007ab0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 10 monclient: shutdown 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e8155f80 msgr2=0x7f85e8151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 --2- 10.20.193.6:0/3222516727 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e8155f80 0x7f85e8151130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85dc02f410 tx=0x7f85dc0047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85ee549640 1 -- 10.20.193.6:0/3222516727 reap_dead start 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 shutdown_connections 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 >> 10.20.193.6:0/3222516727 conn(0x7f85e80cc020 msgr2=0x7f85e815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 shutdown_connections 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 -- 10.20.193.6:0/3222516727 wait complete. 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f85effd3640 1 librados: shutdown 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f443f7fe640 10 client.4434.objecter ms_handle_connect 0x7f442c081390 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.700+0000 7f445f7fe640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7f442c081390 0x7f442c0837d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.14 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.702+0000 7ff674b0c640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff67015d2c0 0x7ff67014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f886d6a0640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f886815f650 0x7f8868156950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f885c00c700 tx=0x7f885c00cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f88567fc640 10 client.?.objecter ms_handle_connect 0x7f886815f650 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f88567fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c80052d0 con 0x7f55dc0cbab0 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: dump: 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c8005470 con 0x7f55dc0cbab0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8003930 con 0x7f55dc0cbab0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 5 monclient: authenticate success, global_id 4444 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 10 monclient: _renew_subs 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1eff10 con 0x7f55dc0cbab0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55da7fc640 4 set_mon_vals no callback set 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55d8ff9640 10 monclient: dump: 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 10 client.4444.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 10 monclient: _renew_subs 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1f04b0 con 0x7f55dc0cbab0 2026-02-22T23:23:27.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7f55e2698640 1 librados: init done 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7ff674b0c640 10 monclient(hunting): get_auth_request con 0x7ff67015d2c0 auth_method 0 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7ff674b0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7ff674b0c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.703+0000 7ff674b0c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7f55e2698640 10 client.4444.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7f55e2698640 15 client.4444.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7f55e2698640 20 client.4444.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7f55e2698640 10 client.4444.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7ff674b0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7ff674b0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7f55e2698640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55dc072f10 con 0x7f55dc0cbab0 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7ff674b0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7fe0f6ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f80d1ea0 0x7fe0f80d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.704+0000 7f65d7fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015f650 0x7f65e00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 10 monclient(hunting): handle_auth_done global_id 4453 payload 824 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 1 -- 10.20.193.6:0/1016129297 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6701527f0 msgr2=0x7ff6701f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff6701527f0 0x7ff6701f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 1 -- 10.20.193.6:0/1016129297 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff67015d730 msgr2=0x7ff6700c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff67015d730 0x7ff6700c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7ff674b0c640 1 -- 10.20.193.6:0/1016129297 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6701f55c0 con 0x7ff67015d2c0 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7fe0f6ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f80d1ea0 0x7fe0f80d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33216/0 (socket says 10.20.193.6:33216) 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7fe0f6ffd640 1 -- 10.20.193.6:0/3017583479 learned_addr learned my addr 10.20.193.6:0/3017583479 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7fe0f6ffd640 10 monclient(hunting): get_auth_request con 0x7fe0f80d1ea0 auth_method 0 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7fe0f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7fe0f6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7fe0f6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7f65d7fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015f650 0x7f65e00c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33218/0 (socket says 10.20.193.6:33218) 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7f65d7fff640 1 -- 10.20.193.6:0/391250990 learned_addr learned my addr 10.20.193.6:0/391250990 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7f65d7fff640 10 monclient(hunting): get_auth_request con 0x7f65e015f650 auth_method 0 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7f65d7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7f65d7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.706+0000 7f65d7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.751 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:27.751 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:23:27.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.708+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8b68003890 con 0x7f8b80155ef0 2026-02-22T23:23:27.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.709+0000 7f8b757fa640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8b50078e50 0x7f8b5007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.709+0000 7f8b84cd1640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8b50078e50 0x7f8b5007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.709+0000 7f8b84cd1640 10 monclient: get_auth_request con 0x7f8b50078e50 auth_method 0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.709+0000 7f8b84cd1640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8b50078e50 0x7f8b5007b310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006fd0 tx=0x7f8b70008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.709+0000 7f8b757fa640 10 client.4463.objecter ms_handle_connect 0x7f8b50078e50 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f8b6805d340 con 0x7f8b80155ef0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 10 client.4463.objecter ms_dispatch2 0x7f8b80000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 3 client.4463.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 3 client.4463.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 10 client.4463.objecter checking command 1 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7f8b500813f0 0x7f8b50083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 20 client.4463.objecter _get_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 20 client.4463.objecter put_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 15 client.4463.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 20 client.4463.objecter _get_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 20 client.4463.objecter get_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 15 client.4463.objecter _session_command_op_assign 9 1 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 20 client.4463.objecter put_session s=0x7f8b50080d20 osd=9 4 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 10 client.4463.objecter _send_command 1 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 --> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b6800bf30 con 0x7f8b500813f0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 20 client.4463.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b6805d700 con 0x7f8b80155ef0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b757fa640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b854d2640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7f8b500813f0 0x7f8b50083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.711+0000 7f8b854d2640 10 monclient: get_auth_request con 0x7f8b500813f0 auth_method 0 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fe0f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fe0f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fe0f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088044070 con 0x7fd09c0ceca0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: dump: 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd088040630 con 0x7fd09c0ceca0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08803c040 con 0x7fd09c0ceca0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 5 monclient: authenticate success, global_id 4443 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd099ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 monclient: _renew_subs 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c1f7d50 con 0x7fd09c0ceca0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd077fff640 10 monclient: dump: 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.753 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 client.4443.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 monclient: _renew_subs 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c1f8310 con 0x7fd09c0ceca0 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 1 librados: init done 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7f65d7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7f65d7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7f65d7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 client.4443.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 15 client.4443.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 20 client.4443.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 client.4443.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd09c1f8510 con 0x7fd09c0ceca0 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7ff674b0c640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff67015d2c0 0x7ff67014c240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff66400b580 tx=0x7ff66400ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7ff65dffb640 10 client.?.objecter ms_handle_connect 0x7ff67015d2c0 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.712+0000 7ff65dffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.713+0000 7f8b854d2640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7f8b500813f0 0x7f8b50083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.713+0000 7f8b757fa640 10 client.4463.objecter ms_handle_connect 0x7f8b500813f0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.714+0000 7f65e4c86640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65e00d1ea0 0x7f65e014c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.714+0000 7fe0f6ffd640 10 monclient(hunting): handle_auth_done global_id 4456 payload 824 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.714+0000 7fe0f6ffd640 1 -- 10.20.193.6:0/3017583479 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe0f815fa30 msgr2=0x7fe0f814cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe0f815fa30 0x7fe0f814cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 1 -- 10.20.193.6:0/3017583479 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0f815f650 msgr2=0x7fe0f814c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0f815f650 0x7fe0f814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7fe0f6ffd640 1 -- 10.20.193.6:0/3017583479 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f81e90d0 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 10 monclient(hunting): handle_auth_done global_id 4459 payload 824 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 1 -- 10.20.193.6:0/391250990 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65e015fa30 msgr2=0x7f65e0149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65e015fa30 0x7f65e0149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 1 -- 10.20.193.6:0/391250990 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65e00d1ea0 msgr2=0x7f65e014c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65e00d1ea0 0x7f65e014c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.715+0000 7f65d7fff640 1 -- 10.20.193.6:0/391250990 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e014b560 con 0x7f65e015f650 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.717+0000 7fe0f6ffd640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f80d1ea0 0x7fe0f80d0e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c700 tx=0x7fe0ec00cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.717+0000 7fe0cffff640 10 client.?.objecter ms_handle_connect 0x7fe0f80d1ea0 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.717+0000 7fe0cffff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.717+0000 7f65d7fff640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015f650 0x7f65e00c9dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65c802f370 tx=0x7f65c80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.717+0000 7f65d57fa640 10 client.?.objecter ms_handle_connect 0x7f65e015f650 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.717+0000 7f65d57fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc26801d030 con 0x7fc27815fa30 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7fc24ffff640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc248078e50 0x7fc24807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7fc276ffd640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc248078e50 0x7fc24807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7fc276ffd640 10 monclient: get_auth_request con 0x7fc248078e50 auth_method 0 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664004500 con 0x7ff67015d2c0 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: dump: 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.756 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff664014070 con 0x7ff67015d2c0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65f7fe640 4 set_mon_vals no callback set 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 5 monclient: authenticate success, global_id 4453 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 10 monclient: _renew_subs 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640029c0 con 0x7ff67015d2c0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6701f7b80 con 0x7ff67015d2c0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff65dffb640 10 monclient: dump: 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 10 monclient: _renew_subs 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6701f3e00 con 0x7ff67015d2c0 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.719+0000 7ff677598640 1 librados: init done 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7fc24ffff640 10 client.4469.objecter ms_handle_connect 0x7fc248078e50 2026-02-22T23:23:27.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7fc276ffd640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc248078e50 0x7fc24807b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc26c006fd0 tx=0x7fc26c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7ff677598640 10 client.4453.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7ff677598640 15 client.4453.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7ff677598640 20 client.4453.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7ff677598640 10 client.4453.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7ff677598640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.720+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff6700d29e0 con 0x7ff67015d2c0 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fccc8041a70 con 0x7fcce0155e70 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fc26805d810 con 0x7fc27815fa30 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 10 client.4469.objecter ms_dispatch2 0x7fc278000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 3 client.4469.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 3 client.4469.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 10 client.4469.objecter checking command 1 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7fc2480813f0 0x7fc248083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 20 client.4469.objecter _get_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 20 client.4469.objecter put_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 15 client.4469.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 20 client.4469.objecter _get_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 20 client.4469.objecter get_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 15 client.4469.objecter _session_command_op_assign 8 1 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 20 client.4469.objecter put_session s=0x7fc248080d20 osd=8 4 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 10 client.4469.objecter _send_command 1 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 --> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc268015e50 con 0x7fc2480813f0 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 20 client.4469.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fccc6ffd640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fccac081890 0x7fccac083d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fccde575640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fccac081890 0x7fccac083d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fccde575640 10 monclient: get_auth_request con 0x7fccac081890 auth_method 0 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc2767fc640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7fc2480813f0 0x7fc248083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc26805dbd0 con 0x7fc27815fa30 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.721+0000 7fc24ffff640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7fc2767fc640 10 monclient: get_auth_request con 0x7fc2480813f0 auth_method 0 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7fccde575640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fccac081890 0x7fccac083d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccd4007fd0 tx=0x7fccd4007b40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7fccc6ffd640 10 client.4419.objecter ms_handle_connect 0x7fccac081890 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0044070 con 0x7ff6000c9b00 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: dump: 2026-02-22T23:23:27.759 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f0035990 con 0x7ff6000c9b00 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f003c040 con 0x7ff6000c9b00 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 5 monclient: authenticate success, global_id 4450 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 monclient: _renew_subs 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5effff640 4 set_mon_vals no callback set 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60014ef70 con 0x7ff6000c9b00 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff5ee7fc640 10 monclient: dump: 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 monclient: _renew_subs 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60014f530 con 0x7ff6000c9b00 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 1 librados: init done 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 client.4450.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 15 client.4450.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 20 client.4450.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 client.4450.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.722+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff6000cc020 con 0x7ff6000c9b00 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.724+0000 7fc2767fc640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7fc2480813f0 0x7fc248083830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.724+0000 7fc24ffff640 10 client.4469.objecter ms_handle_connect 0x7fc2480813f0 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.729+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fccc802f9c0 con 0x7fcce0155e70 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.729+0000 7fccc6ffd640 10 client.4419.objecter ms_dispatch2 0x7fcce0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.729+0000 7fccc6ffd640 3 client.4419.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.729+0000 7fccc6ffd640 3 client.4419.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 10 client.4419.objecter checking command 1 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7fccac089dd0 0x7fccac08c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 20 client.4419.objecter _get_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 20 client.4419.objecter put_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 15 client.4419.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 20 client.4419.objecter _get_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 20 client.4419.objecter get_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 15 client.4419.objecter _session_command_op_assign 4 1 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 20 client.4419.objecter put_session s=0x7fccac089700 osd=4 4 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 10 client.4419.objecter _send_command 1 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 --> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccac08c8e0 con 0x7fccac089dd0 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 20 client.4419.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccc80835b0 con 0x7fcce0155e70 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccc6ffd640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4d3c03f050 con 0x7f4d4c1527f0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccded76640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7fccac089dd0 0x7fccac08c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f55c8003ad0 con 0x7f55dc0cbab0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7f4d417fa640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4d1c078e50 0x7f4d1c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7fccded76640 10 monclient: get_auth_request con 0x7fccac089dd0 auth_method 0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7f4d43fff640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4d1c078e50 0x7f4d1c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.730+0000 7f4d43fff640 10 monclient: get_auth_request con 0x7f4d1c078e50 auth_method 0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f55d8ff9640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f55ac078e80 0x7f55ac07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d43fff640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4d1c078e50 0x7f4d1c07b310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006b90 tx=0x7f4d34006b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 10 client.4466.objecter ms_handle_connect 0x7f4d1c078e50 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f55db7fe640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f55ac078e80 0x7f55ac07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f55db7fe640 10 monclient: get_auth_request con 0x7f55ac078e80 auth_method 0 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f4d3c0bc600 con 0x7f4d4c1527f0 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 10 client.4466.objecter ms_dispatch2 0x7f4d4c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 3 client.4466.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 3 client.4466.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 10 client.4466.objecter checking command 1 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f4d1c081390 0x7f4d1c0837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 20 client.4466.objecter _get_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 20 client.4466.objecter put_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 15 client.4466.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 20 client.4466.objecter _get_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 20 client.4466.objecter get_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 15 client.4466.objecter _session_command_op_assign 0 1 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 20 client.4466.objecter put_session s=0x7f4d1c080cc0 osd=0 4 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 10 client.4466.objecter _send_command 1 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 --> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d3c04ee80 con 0x7f4d1c081390 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f4d417fa640 20 client.4466.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f78005bb0 con 0x7f5f881527f0 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 10 monclient: dump: 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f55d8ff9640 10 client.4444.objecter ms_handle_connect 0x7f55ac078e80 2026-02-22T23:23:27.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f55db7fe640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f55ac078e80 0x7f55ac07b340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f55cc039000 tx=0x7f55cc033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f78005d50 con 0x7f5f881527f0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.731+0000 7f5f8e144640 5 monclient: authenticate success, global_id 4440 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 monclient: _renew_subs 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f880c68f0 con 0x7f5f881527f0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f63fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f780072b0 con 0x7f5f881527f0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f63fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f63fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f63fff640 10 monclient: dump: 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f85ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 client.4440.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 monclient: _renew_subs 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f880c6eb0 con 0x7f5f881527f0 2026-02-22T23:23:27.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 1 librados: init done 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c0832f0 con 0x7f4d4c1527f0 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f4d417fa640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f55c805dea0 con 0x7f55dc0cbab0 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 10 client.4444.objecter ms_dispatch2 0x7f55dc000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 3 client.4444.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 3 client.4444.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f4d51605640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f4d1c081390 0x7f4d1c0837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 10 client.4444.objecter checking command 1 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f55ac0813c0 0x7f55ac083800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 20 client.4444.objecter _get_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 20 client.4444.objecter put_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 15 client.4444.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7fccded76640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7fccac089dd0 0x7fccac08c210 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7fccc6ffd640 10 client.4419.objecter ms_handle_connect 0x7fccac089dd0 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 20 client.4444.objecter _get_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 20 client.4444.objecter get_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 15 client.4444.objecter _session_command_op_assign 12 1 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 20 client.4444.objecter put_session s=0x7f55ac080cf0 osd=12 4 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 10 client.4444.objecter _send_command 1 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 --> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f55c8010e50 con 0x7f55ac0813c0 2026-02-22T23:23:27.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f55d8ff9640 20 client.4444.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f4d51605640 10 monclient: get_auth_request con 0x7f4d1c081390 auth_method 0 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 client.4440.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 15 client.4440.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 20 client.4440.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 client.4440.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.732+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f88074fa0 con 0x7f5f881527f0 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f55c805e260 con 0x7f55dc0cbab0 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7f55d8ff9640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7f55e0c0e640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f55ac0813c0 0x7f55ac083800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7f55e0c0e640 10 monclient: get_auth_request con 0x7f55ac0813c0 auth_method 0 2026-02-22T23:23:27.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370046020 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: dump: 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370041450 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037e60 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc35affd640 10 monclient: dump: 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc378cf0640 4 set_mon_vals no callback set 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 5 monclient: authenticate success, global_id 4478 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 10 monclient: _renew_subs 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36c0d1560 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 10 client.4478.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 10 monclient: _renew_subs 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36c0d1b20 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.733+0000 7fc37b4f5640 1 librados: init done 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7fc37b4f5640 10 client.4478.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7fc37b4f5640 15 client.4478.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7fc37b4f5640 20 client.4478.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7fc37b4f5640 10 client.4478.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7fc37b4f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc36c008770 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7f4d51605640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f4d1c081390 0x7f4d1c0837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.734+0000 7f4d417fa640 10 client.4466.objecter ms_handle_connect 0x7f4d1c081390 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.735+0000 7f55d8ff9640 10 client.4444.objecter ms_handle_connect 0x7f55ac0813c0 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.735+0000 7f55e0c0e640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f55ac0813c0 0x7f55ac083800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.12 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c017070 con 0x7f886815f650 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: dump: 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.768 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885c005bb0 con 0x7f886815f650 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c007070 con 0x7f886815f650 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f887012c640 5 monclient: authenticate success, global_id 4447 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f88567fc640 10 monclient: dump: 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f887012c640 10 monclient: _renew_subs 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f8857fff640 4 set_mon_vals no callback set 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f887012c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.736+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886814c010 con 0x7f886815f650 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 10 client.4447.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 10 monclient: _renew_subs 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8868159770 con 0x7f886815f650 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 1 librados: init done 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== osd.14 v2:10.20.193.80:6824/3509265050 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f442c083ea0 con 0x7f442c081390 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f443f7fe640 10 client.4434.objecter ms_dispatch2 0x7f4460000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f443f7fe640 10 client.4434.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f443f7fe640 20 client.4434.objecter put_session s=0x7f442c080cc0 osd=14 4 2026-02-22T23:23:27.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f443f7fe640 15 client.4434.objecter _session_command_op_remove 14 1 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 10 client.4447.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 15 client.4447.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 20 client.4447.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 10 client.4447.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.737+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8868074ea0 con 0x7f886815f650 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8044070 con 0x7f65e015f650 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: dump: 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c8040630 con 0x7f65e015f650 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c803c040 con 0x7f65e015f650 2026-02-22T23:23:27.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 5 monclient: authenticate success, global_id 4459 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d6ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65d57fa640 10 monclient: dump: 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 monclient: _renew_subs 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e014b840 con 0x7f65e015f650 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 client.4459.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 monclient: _renew_subs 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e014be00 con 0x7f65e015f650 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 1 librados: init done 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== osd.9 v2:10.20.193.40:6808/979607499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f8b6800bf30 con 0x7f8b500813f0 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f8b757fa640 10 client.4463.objecter ms_dispatch2 0x7f8b80000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f8b757fa640 10 client.4463.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f8b757fa640 20 client.4463.objecter put_session s=0x7f8b50080d20 osd=9 4 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f8b757fa640 15 client.4463.objecter _session_command_op_remove 9 1 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 client.4459.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 15 client.4459.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 20 client.4459.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 client.4459.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.738+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65e014c6a0 con 0x7f65e015f650 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.739+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe5f803f050 con 0x7fe60c0c79e0 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.740+0000 7fe5e7fff640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe5dc078e50 0x7fe5dc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.740+0000 7fe60affd640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe5dc078e50 0x7fe5dc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.740+0000 7fe60affd640 10 monclient: get_auth_request con 0x7fe5dc078e50 auth_method 0 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.741+0000 7fe60affd640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe5dc078e50 0x7fe5dc07b310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe60c15c600 tx=0x7fe5fc006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.741+0000 7fe5e7fff640 10 client.4425.objecter ms_handle_connect 0x7fe5dc078e50 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fe5f80830a0 con 0x7fe60c0c79e0 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 10 client.4425.objecter ms_dispatch2 0x7fe60c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 3 client.4425.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 3 client.4425.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 10 client.4425.objecter checking command 1 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fe5dc081390 0x7fe5dc0837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 20 client.4425.objecter _get_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 20 client.4425.objecter put_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 15 client.4425.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 20 client.4425.objecter _get_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 20 client.4425.objecter get_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 15 client.4425.objecter _session_command_op_assign 11 1 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 20 client.4425.objecter put_session s=0x7fe5dc080cc0 osd=11 4 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 10 client.4425.objecter _send_command 1 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 --> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5f8048ec0 con 0x7fe5dc081390 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 20 client.4425.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5f8083460 con 0x7fe60c0c79e0 2026-02-22T23:23:27.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.743+0000 7fe5e7fff640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 10 client.4434.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 15 client.4434.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 20 client.4434.objecter _get_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 20 client.4434.objecter put_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 20 client.4434.objecter _get_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 15 client.4434.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 20 client.4434.objecter get_session s=0x7f442c080cc0 osd=14 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 15 client.4434.objecter _session_command_op_assign 14 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 20 client.4434.objecter put_session s=0x7f442c080cc0 osd=14 4 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 10 client.4434.objecter _send_command 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 --> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44601608b0 con 0x7f442c081390 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7fe60bfff640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fe5dc081390 0x7fe5dc0837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.744+0000 7fe60bfff640 10 monclient: get_auth_request con 0x7fe5dc081390 auth_method 0 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 10 client.4463.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 15 client.4463.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 20 client.4463.objecter _get_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 20 client.4463.objecter put_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 20 client.4463.objecter _get_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 15 client.4463.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 20 client.4463.objecter get_session s=0x7f8b50080d20 osd=9 3 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 15 client.4463.objecter _session_command_op_assign 9 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 20 client.4463.objecter put_session s=0x7f8b50080d20 osd=9 4 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 10 client.4463.objecter _send_command 2 2026-02-22T23:23:27.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 --> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b80077010 con 0x7f8b500813f0 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== osd.8 v2:10.20.193.6:6816/1002652796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fc268015e50 con 0x7fc2480813f0 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fc24ffff640 10 client.4469.objecter ms_dispatch2 0x7fc278000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fc24ffff640 10 client.4469.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fc24ffff640 20 client.4469.objecter put_session s=0x7fc248080d20 osd=8 4 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fc24ffff640 15 client.4469.objecter _session_command_op_remove 8 1 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0052d0 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: dump: 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0ec005470 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00a860 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 5 monclient: authenticate success, global_id 4456 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0f57fa640 4 set_mon_vals no callback set 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 10 monclient: _renew_subs 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f81e93b0 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0cffff640 10 monclient: dump: 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 10 client.4456.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 10 monclient: _renew_subs 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f81e9970 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.745+0000 7fe0fda1d640 1 librados: init done 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff664019460 con 0x7ff67015d2c0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7ff65dffb640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff640078e50 0x7ff64007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7fe0fda1d640 10 client.4456.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7fe0fda1d640 15 client.4456.objecter _session_command_op_assign -1 1 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7fe0fda1d640 20 client.4456.objecter _get_session osd=-1 returning homeless 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7fe0fda1d640 10 client.4456.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7fe0fda1d640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0f80d1960 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7ff67530d640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff640078e50 0x7ff64007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7ff67530d640 10 monclient: get_auth_request con 0x7ff640078e50 auth_method 0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7ff67530d640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff640078e50 0x7ff64007b310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff660009380 tx=0x7ff660033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.746+0000 7ff65dffb640 10 client.4453.objecter ms_handle_connect 0x7ff640078e50 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.747+0000 7fe60bfff640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fe5dc081390 0x7fe5dc0837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.747+0000 7fe5e7fff640 10 client.4425.objecter ms_handle_connect 0x7fe5dc081390 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.747+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc370041840 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.747+0000 7fc35affd640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc344078e50 0x7fc34407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.747+0000 7fc37a4f3640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc344078e50 0x7fc34407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.747+0000 7fc37a4f3640 10 monclient: get_auth_request con 0x7fc344078e50 auth_method 0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc37a4f3640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc344078e50 0x7fc34407b310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc36400a8d0 tx=0x7fc364008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 10 client.4478.objecter ms_handle_connect 0x7fc344078e50 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fc370083f50 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 10 client.4478.objecter ms_dispatch2 0x7fc374000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 3 client.4478.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 3 client.4478.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 10 client.4478.objecter checking command 1 2026-02-22T23:23:27.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7fc344081390 0x7fc3440837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 20 client.4478.objecter _get_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 20 client.4478.objecter put_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 15 client.4478.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 20 client.4478.objecter _get_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 20 client.4478.objecter get_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 15 client.4478.objecter _session_command_op_assign 7 1 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 20 client.4478.objecter put_session s=0x7fc344080cc0 osd=7 4 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 10 client.4478.objecter _send_command 1 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 --> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc344083ea0 con 0x7fc344081390 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 20 client.4478.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc370084310 con 0x7fc36c0e0fa0 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7fc35affd640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7ff664097630 con 0x7ff67015d2c0 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 10 client.4453.objecter ms_dispatch2 0x7ff670000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 3 client.4453.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 3 client.4453.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 10 client.4453.objecter checking command 1 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7ff640081390 0x7ff6400837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 20 client.4453.objecter _get_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 20 client.4453.objecter put_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 15 client.4453.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 20 client.4453.objecter _get_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 20 client.4453.objecter get_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 15 client.4453.objecter _session_command_op_assign 15 1 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 20 client.4453.objecter put_session s=0x7ff640080cc0 osd=15 4 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 10 client.4453.objecter _send_command 1 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 --> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff640083ea0 con 0x7ff640081390 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.748+0000 7ff65dffb640 20 client.4453.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff66405e150 con 0x7ff67015d2c0 2026-02-22T23:23:27.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7ff65dffb640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b757fa640 1 -- 10.20.193.6:0/3080835411 <== osd.9 v2:10.20.193.40:6808/979607499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8b80077010 con 0x7f8b500813f0 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b757fa640 10 client.4463.objecter ms_dispatch2 0x7f8b80000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b757fa640 10 client.4463.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b757fa640 20 client.4463.objecter put_session s=0x7f8b50080d20 osd=9 4 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b757fa640 15 client.4463.objecter _session_command_op_remove 9 2 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7fc379cf2640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7fc344081390 0x7fc3440837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7ff675b0e640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7ff640081390 0x7ff6400837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7ff675b0e640 10 monclient: get_auth_request con 0x7ff640081390 auth_method 0 2026-02-22T23:23:27.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7fc379cf2640 10 monclient: get_auth_request con 0x7fc344081390 auth_method 0 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 10 librados: watch_flush enter 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 10 librados: watch_flush exit 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 10 client.4463.objecter close_session for osd.9 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7f8b500813f0 msgr2=0x7f8b50083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7f8b500813f0 0x7f8b50083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 20 client.4463.objecter put_session s=0x7f8b50080d20 osd=9 1 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 20 client.4463.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 10 client.4463.objecter successfully canceled tick 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8b50078e50 msgr2=0x7f8b5007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8b50078e50 0x7f8b5007b310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b70006fd0 tx=0x7f8b70008040 comp rx=0 tx=0).stop 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 10 monclient: shutdown 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b80155ef0 msgr2=0x7f8b801f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 --2- 10.20.193.6:0/3080835411 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8b80155ef0 0x7f8b801f3b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800c480 tx=0x7f8b6800c950 comp rx=0 tx=0).stop 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b854d2640 1 -- 10.20.193.6:0/3080835411 reap_dead start 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 shutdown_connections 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 >> 10.20.193.6:0/3080835411 conn(0x7f8b8006d8c0 msgr2=0x7f8b8006dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 shutdown_connections 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 -- 10.20.193.6:0/3080835411 wait complete. 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.749+0000 7f8b86f5c640 1 librados: shutdown 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f443f7fe640 1 -- 10.20.193.6:0/1220287504 <== osd.14 v2:10.20.193.80:6824/3509265050 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f44601608b0 con 0x7f442c081390 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f443f7fe640 10 client.4434.objecter ms_dispatch2 0x7f4460000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f443f7fe640 10 client.4434.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f443f7fe640 20 client.4434.objecter put_session s=0x7f442c080cc0 osd=14 4 2026-02-22T23:23:27.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f443f7fe640 15 client.4434.objecter _session_command_op_remove 14 2 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 10 librados: watch_flush enter 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 10 librados: watch_flush exit 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 10 client.4434.objecter close_session for osd.14 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7f442c081390 msgr2=0x7f442c0837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7f442c081390 0x7f442c0837d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 20 client.4434.objecter put_session s=0x7f442c080cc0 osd=14 1 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 20 client.4434.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 10 client.4434.objecter successfully canceled tick 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f442c078e50 msgr2=0x7f442c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f442c078e50 0x7f442c07b310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4448007fd0 tx=0x7f4448007b20 comp rx=0 tx=0).stop 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 10 monclient: shutdown 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f44600ca160 msgr2=0x7f4460153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 --2- 10.20.193.6:0/1220287504 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f44600ca160 0x7f4460153b10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f445400c700 tx=0x7f445400cbd0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f445f7fe640 1 -- 10.20.193.6:0/1220287504 reap_dead start 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 shutdown_connections 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 >> 10.20.193.6:0/1220287504 conn(0x7f446006d8e0 msgr2=0x7f446006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 shutdown_connections 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.750+0000 7f4465632640 1 -- 10.20.193.6:0/1220287504 wait complete. 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.751+0000 7f4465632640 1 librados: shutdown 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.751+0000 7fc379cf2640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7fc344081390 0x7fc3440837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.751+0000 7fc35affd640 10 client.4478.objecter ms_handle_connect 0x7fc344081390 2026-02-22T23:23:27.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.751+0000 7ff65dffb640 10 client.4453.objecter ms_handle_connect 0x7ff640081390 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.751+0000 7ff675b0e640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7ff640081390 0x7ff6400837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.15 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 10 client.4469.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 15 client.4469.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 20 client.4469.objecter _get_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 20 client.4469.objecter put_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 20 client.4469.objecter _get_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 15 client.4469.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 20 client.4469.objecter get_session s=0x7fc248080d20 osd=8 3 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 15 client.4469.objecter _session_command_op_assign 8 2 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 20 client.4469.objecter put_session s=0x7fc248080d20 osd=8 4 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 10 client.4469.objecter _send_command 2 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.752+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 --> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc27814a7b0 con 0x7fc2480813f0 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.754+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== osd.4 v2:10.20.193.6:6800/2180113666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fccac08c8e0 con 0x7fccac089dd0 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.754+0000 7fccc6ffd640 10 client.4419.objecter ms_dispatch2 0x7fcce0000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.754+0000 7fccc6ffd640 10 client.4419.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.754+0000 7fccc6ffd640 20 client.4419.objecter put_session s=0x7fccac089700 osd=4 4 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.754+0000 7fccc6ffd640 15 client.4419.objecter _session_command_op_remove 4 1 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== osd.0 v2:10.20.193.6:6824/1913492598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f4d3c04ee80 con 0x7f4d1c081390 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7f4d417fa640 10 client.4466.objecter ms_dispatch2 0x7f4d4c000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7f4d417fa640 10 client.4466.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7f4d417fa640 20 client.4466.objecter put_session s=0x7f4d1c080cc0 osd=0 4 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7f4d417fa640 15 client.4466.objecter _session_command_op_remove 0 1 2026-02-22T23:23:27.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7fc24ffff640 1 -- 10.20.193.6:0/268222076 <== osd.8 v2:10.20.193.6:6816/1002652796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc27814a7b0 con 0x7fc2480813f0 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7fc24ffff640 10 client.4469.objecter ms_dispatch2 0x7fc278000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7fc24ffff640 10 client.4469.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7fc24ffff640 20 client.4469.objecter put_session s=0x7fc248080d20 osd=8 4 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.756+0000 7fc24ffff640 15 client.4469.objecter _session_command_op_remove 8 2 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 10 librados: watch_flush enter 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 10 librados: watch_flush exit 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 10 client.4469.objecter close_session for osd.8 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7fc2480813f0 msgr2=0x7fc248083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7fc2480813f0 0x7fc248083830 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 20 client.4469.objecter put_session s=0x7fc248080d20 osd=8 1 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 20 client.4469.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 10 client.4469.objecter successfully canceled tick 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc248078e50 msgr2=0x7fc24807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc248078e50 0x7fc24807b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc26c006fd0 tx=0x7fc26c008040 comp rx=0 tx=0).stop 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 10 monclient: shutdown 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 msgr2=0x7fc2781f90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 --2- 10.20.193.6:0/268222076 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc27815fa30 0x7fc2781f90a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc26800b970 tx=0x7fc26800be40 comp rx=0 tx=0).stop 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc2777fe640 1 -- 10.20.193.6:0/268222076 reap_dead start 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 shutdown_connections 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 >> 10.20.193.6:0/268222076 conn(0x7fc278072fc0 msgr2=0x7fc27815cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 shutdown_connections 2026-02-22T23:23:27.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 -- 10.20.193.6:0/268222076 wait complete. 2026-02-22T23:23:27.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.757+0000 7fc27da43640 1 librados: shutdown 2026-02-22T23:23:27.783 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:27.783 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-22T23:23:27.783 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:27.784 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 10 client.4419.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 15 client.4419.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 20 client.4419.objecter _get_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 20 client.4419.objecter put_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 20 client.4419.objecter _get_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 15 client.4419.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 20 client.4419.objecter get_session s=0x7fccac089700 osd=4 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 15 client.4419.objecter _session_command_op_assign 4 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 20 client.4419.objecter put_session s=0x7fccac089700 osd=4 4 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 10 client.4419.objecter _send_command 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.761+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 --> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcce0159890 con 0x7fccac089dd0 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 10 client.4466.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 15 client.4466.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 20 client.4466.objecter _get_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 20 client.4466.objecter put_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 20 client.4466.objecter _get_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 15 client.4466.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 20 client.4466.objecter get_session s=0x7f4d1c080cc0 osd=0 3 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 15 client.4466.objecter _session_command_op_assign 0 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 20 client.4466.objecter put_session s=0x7f4d1c080cc0 osd=0 4 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 10 client.4466.objecter _send_command 2 2026-02-22T23:23:27.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.763+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 --> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d4c0caf80 con 0x7f4d1c081390 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:27.785 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.765+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd08803f050 con 0x7fd09c0ceca0 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.765+0000 7fd077fff640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd06c078e80 0x7fd06c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fd09affd640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd06c078e80 0x7fd06c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fd09affd640 10 monclient: get_auth_request con 0x7fd06c078e80 auth_method 0 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fccc6ffd640 1 -- 10.20.193.6:0/4269421126 <== osd.4 v2:10.20.193.6:6800/2180113666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcce0159890 con 0x7fccac089dd0 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fccc6ffd640 10 client.4419.objecter ms_dispatch2 0x7fcce0000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fccc6ffd640 10 client.4419.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fccc6ffd640 20 client.4419.objecter put_session s=0x7fccac089700 osd=4 4 2026-02-22T23:23:27.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fccc6ffd640 15 client.4419.objecter _session_command_op_remove 4 2 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 10 librados: watch_flush enter 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 10 librados: watch_flush exit 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 10 client.4419.objecter close_session for osd.4 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7fccac089dd0 msgr2=0x7fccac08c210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7fccac089dd0 0x7fccac08c210 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 20 client.4419.objecter put_session s=0x7fccac089700 osd=4 1 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 20 client.4419.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 10 client.4419.objecter successfully canceled tick 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fccac081890 msgr2=0x7fccac083d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fccac081890 0x7fccac083d50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccd4007fd0 tx=0x7fccd4007b40 comp rx=0 tx=0).stop 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 10 monclient: shutdown 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcce0155e70 msgr2=0x7fcce00c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 --2- 10.20.193.6:0/4269421126 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcce0155e70 0x7fcce00c6c70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccc802f3a0 tx=0x7fccc80047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fccded76640 1 -- 10.20.193.6:0/4269421126 reap_dead start 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 shutdown_connections 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 >> 10.20.193.6:0/4269421126 conn(0x7fcce00ce700 msgr2=0x7fcce015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fd077fff640 10 client.4443.objecter ms_handle_connect 0x7fd06c078e80 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fd09affd640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd06c078e80 0x7fd06c07b340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0c8c40 tx=0x7fd08c006cf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 shutdown_connections 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 -- 10.20.193.6:0/4269421126 wait complete. 2026-02-22T23:23:27.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7fcce4f1b640 1 librados: shutdown 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d417fa640 1 -- 10.20.193.6:0/1771385440 <== osd.0 v2:10.20.193.6:6824/1913492598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4d4c0caf80 con 0x7f4d1c081390 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d417fa640 10 client.4466.objecter ms_dispatch2 0x7f4d4c000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d417fa640 10 client.4466.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d417fa640 20 client.4466.objecter put_session s=0x7f4d1c080cc0 osd=0 4 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d417fa640 15 client.4466.objecter _session_command_op_remove 0 2 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 10 librados: watch_flush enter 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 10 librados: watch_flush exit 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 10 client.4466.objecter close_session for osd.0 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f4d1c081390 msgr2=0x7f4d1c0837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f4d1c081390 0x7f4d1c0837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 20 client.4466.objecter put_session s=0x7f4d1c080cc0 osd=0 1 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 20 client.4466.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 10 client.4466.objecter successfully canceled tick 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f885c007210 con 0x7f886815f650 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4d1c078e50 msgr2=0x7f4d1c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4d1c078e50 0x7f4d1c07b310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006b90 tx=0x7f4d34006b20 comp rx=0 tx=0).stop 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 10 monclient: shutdown 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.766+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4d4c1527f0 msgr2=0x7f4d4c0c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f4d5308f640 1 --2- 10.20.193.6:0/1771385440 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4d4c1527f0 0x7f4d4c0c9000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c002410 tx=0x7f4d3c0047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f4d51605640 1 -- 10.20.193.6:0/1771385440 reap_dead start 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 shutdown_connections 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 >> 10.20.193.6:0/1771385440 conn(0x7f4d4c0ce730 msgr2=0x7f4d4c0cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 shutdown_connections 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f4d5308f640 1 -- 10.20.193.6:0/1771385440 wait complete. 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f4d5308f640 1 librados: shutdown 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f88567fc640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8838078e80 0x7f883807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f886dea1640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8838078e80 0x7f883807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f886dea1640 10 monclient: get_auth_request con 0x7f8838078e80 auth_method 0 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f886dea1640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8838078e80 0x7f883807b340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8858009750 tx=0x7f8858006ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.767+0000 7f88567fc640 10 client.4447.objecter ms_handle_connect 0x7f8838078e80 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.769+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== osd.12 v2:10.20.193.6:6808/138679770 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f55c8010e50 con 0x7f55ac0813c0 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.769+0000 7f55d8ff9640 10 client.4444.objecter ms_dispatch2 0x7f55dc000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.770+0000 7f55d8ff9640 10 client.4444.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.770+0000 7f55d8ff9640 20 client.4444.objecter put_session s=0x7f55ac080cf0 osd=12 4 2026-02-22T23:23:27.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.770+0000 7f55d8ff9640 15 client.4444.objecter _session_command_op_remove 12 1 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.772+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5f78007450 con 0x7f5f881527f0 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.773+0000 7f5f63fff640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5f58078e50 0x7f5f5807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.773+0000 7f5f877fe640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5f58078e50 0x7f5f5807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.773+0000 7f5f877fe640 10 monclient: get_auth_request con 0x7f5f58078e50 auth_method 0 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.773+0000 7f5f877fe640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5f58078e50 0x7f5f5807b310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006fd0 tx=0x7f5f74006d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.773+0000 7f5f63fff640 10 client.4440.objecter ms_handle_connect 0x7f5f58078e50 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stdout:98784247812 2026-02-22T23:23:27.789 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:23:27.789 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:27.790 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== osd.7 v2:10.20.193.189:6810/980553521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fc344083ea0 con 0x7fc344081390 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7fc35affd640 10 client.4478.objecter ms_dispatch2 0x7fc374000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7fc35affd640 10 client.4478.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7fc35affd640 20 client.4478.objecter put_session s=0x7fc344080cc0 osd=7 4 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7fc35affd640 15 client.4478.objecter _session_command_op_remove 7 1 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 10 client.4444.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 15 client.4444.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 20 client.4444.objecter _get_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 20 client.4444.objecter put_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 20 client.4444.objecter _get_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 15 client.4444.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 20 client.4444.objecter get_session s=0x7f55ac080cf0 osd=12 3 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 15 client.4444.objecter _session_command_op_assign 12 2 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 20 client.4444.objecter put_session s=0x7f55ac080cf0 osd=12 4 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 10 client.4444.objecter _send_command 2 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.776+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 --> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f55dc0c67f0 con 0x7f55ac0813c0 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.777+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff5f003f050 con 0x7ff6000c9b00 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.777+0000 7ff5ee7fc640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5d0078e50 0x7ff5d007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.777+0000 7ff605241640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5d0078e50 0x7ff5d007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.777+0000 7ff605241640 10 monclient: get_auth_request con 0x7ff5d0078e50 auth_method 0 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.778+0000 7ff5ee7fc640 10 client.4450.objecter ms_handle_connect 0x7ff5d0078e50 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.778+0000 7ff605241640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5d0078e50 0x7ff5d007b310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff6001ec490 tx=0x7ff5f4009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f885c05d6f0 con 0x7f886815f650 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 10 client.4447.objecter ms_dispatch2 0x7f8868000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 3 client.4447.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 3 client.4447.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 10 client.4447.objecter checking command 1 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f8838081420 0x7f8838083860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 20 client.4447.objecter _get_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 20 client.4447.objecter put_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 15 client.4447.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 20 client.4447.objecter _get_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 20 client.4447.objecter get_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 15 client.4447.objecter _session_command_op_assign 3 1 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 20 client.4447.objecter put_session s=0x7f8838080d50 osd=3 4 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 10 client.4447.objecter _send_command 1 2026-02-22T23:23:27.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8838083f30 con 0x7f8838081420 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f88567fc640 20 client.4447.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.780+0000 7f886e6a2640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f8838081420 0x7f8838083860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f886e6a2640 10 monclient: get_auth_request con 0x7f8838081420 auth_method 0 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fd088082ff0 con 0x7fd09c0ceca0 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 10 client.4443.objecter ms_dispatch2 0x7fd09c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 3 client.4443.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 3 client.4443.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 10 client.4443.objecter checking command 1 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd06c0813c0 0x7fd06c083800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 20 client.4443.objecter _get_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 20 client.4443.objecter put_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 15 client.4443.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 20 client.4443.objecter _get_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 20 client.4443.objecter get_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 15 client.4443.objecter _session_command_op_assign 2 1 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 20 client.4443.objecter put_session s=0x7fd06c080cf0 osd=2 4 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 10 client.4443.objecter _send_command 1 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 --> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd08802fa90 con 0x7fd06c0813c0 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 20 client.4443.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd0880833b0 con 0x7fd09c0ceca0 2026-02-22T23:23:27.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd077fff640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f5f78007770 con 0x7f5f881527f0 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 10 client.4440.objecter ms_dispatch2 0x7f5f88000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 3 client.4440.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 3 client.4440.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 10 client.4440.objecter checking command 1 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7f5f58081390 0x7f5f580837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 20 client.4440.objecter _get_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 20 client.4440.objecter put_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 15 client.4440.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 20 client.4440.objecter _get_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 20 client.4440.objecter get_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 15 client.4440.objecter _session_command_op_assign 10 1 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 20 client.4440.objecter put_session s=0x7f5f58080cc0 osd=10 4 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 10 client.4440.objecter _send_command 1 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 --> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f58083ea0 con 0x7f5f58081390 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 20 client.4440.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f7805e4e0 con 0x7f5f881527f0 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7f5f63fff640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.781+0000 7fd09bfff640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd06c0813c0 0x7fd06c083800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7f5f87fff640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7f5f58081390 0x7f5f580837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7fd09bfff640 10 monclient: get_auth_request con 0x7fd06c0813c0 auth_method 0 2026-02-22T23:23:27.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7f5f87fff640 10 monclient: get_auth_request con 0x7f5f58081390 auth_method 0 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7ff5f002fa10 con 0x7ff6000c9b00 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 10 client.4450.objecter ms_dispatch2 0x7ff600000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 3 client.4450.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 3 client.4450.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7f886e6a2640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f8838081420 0x7f8838083860 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7f88567fc640 10 client.4447.objecter ms_handle_connect 0x7f8838081420 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 10 client.4450.objecter checking command 1 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7ff5d0081390 0x7ff5d00837d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 20 client.4450.objecter _get_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 20 client.4450.objecter put_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 15 client.4450.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 20 client.4450.objecter _get_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 20 client.4450.objecter get_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 15 client.4450.objecter _session_command_op_assign 6 1 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 20 client.4450.objecter put_session s=0x7ff5d0080cc0 osd=6 4 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 10 client.4450.objecter _send_command 1 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 --> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5d0083ea0 con 0x7ff5d0081390 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.782+0000 7ff5ee7fc640 20 client.4450.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55d8ff9640 1 -- 10.20.193.6:0/3130673925 <== osd.12 v2:10.20.193.6:6808/138679770 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f55dc0c67f0 con 0x7f55ac0813c0 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55d8ff9640 10 client.4444.objecter ms_dispatch2 0x7f55dc000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55d8ff9640 10 client.4444.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55d8ff9640 20 client.4444.objecter put_session s=0x7f55ac080cf0 osd=12 4 2026-02-22T23:23:27.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55d8ff9640 15 client.4444.objecter _session_command_op_remove 12 2 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 10 librados: watch_flush enter 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 10 librados: watch_flush exit 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 10 client.4478.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 15 client.4478.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 20 client.4478.objecter _get_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 20 client.4478.objecter put_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 20 client.4478.objecter _get_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 10 client.4444.objecter close_session for osd.12 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f55ac0813c0 msgr2=0x7f55ac083800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f55ac0813c0 0x7f55ac083800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 20 client.4444.objecter put_session s=0x7f55ac080cf0 osd=12 1 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 20 client.4444.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 15 client.4478.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 20 client.4478.objecter get_session s=0x7fc344080cc0 osd=7 3 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 15 client.4478.objecter _session_command_op_assign 7 2 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 20 client.4478.objecter put_session s=0x7fc344080cc0 osd=7 4 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 10 client.4478.objecter _send_command 2 2026-02-22T23:23:27.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 --> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc36c0e02d0 con 0x7fc344081390 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 10 client.4444.objecter successfully canceled tick 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f55ac078e80 msgr2=0x7f55ac07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f55ac078e80 0x7f55ac07b340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f55cc039000 tx=0x7f55cc033000 comp rx=0 tx=0).stop 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 10 monclient: shutdown 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f55dc0cbab0 msgr2=0x7f55dc1e6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 --2- 10.20.193.6:0/3130673925 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f55dc0cbab0 0x7f55dc1e6620 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f55c800c430 tx=0x7f55c800c900 comp rx=0 tx=0).stop 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e0c0e640 1 -- 10.20.193.6:0/3130673925 reap_dead start 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7ff606243640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7ff5d0081390 0x7ff5d00837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 shutdown_connections 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 >> 10.20.193.6:0/3130673925 conn(0x7f55dc0cc190 msgr2=0x7f55dc15c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 shutdown_connections 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 -- 10.20.193.6:0/3130673925 wait complete. 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f55e2698640 1 librados: shutdown 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f885c05dab0 con 0x7f886815f650 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7f88567fc640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5f00836f0 con 0x7ff6000c9b00 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7ff5ee7fc640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.783+0000 7ff606243640 10 monclient: get_auth_request con 0x7ff5d0081390 auth_method 0 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.784+0000 7fd077fff640 10 client.4443.objecter ms_handle_connect 0x7fd06c0813c0 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.784+0000 7fd09bfff640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd06c0813c0 0x7fd06c083800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.784+0000 7f5f63fff640 10 client.4440.objecter ms_handle_connect 0x7f5f58081390 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.784+0000 7f5f87fff640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7f5f58081390 0x7f5f580837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.785+0000 7ff606243640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7ff5d0081390 0x7ff5d00837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.785+0000 7ff5ee7fc640 10 client.4450.objecter ms_handle_connect 0x7ff5d0081390 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc35affd640 1 -- 10.20.193.6:0/996581608 <== osd.7 v2:10.20.193.189:6810/980553521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc36c0e02d0 con 0x7fc344081390 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc35affd640 10 client.4478.objecter ms_dispatch2 0x7fc374000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc35affd640 10 client.4478.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc35affd640 20 client.4478.objecter put_session s=0x7fc344080cc0 osd=7 4 2026-02-22T23:23:27.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc35affd640 15 client.4478.objecter _session_command_op_remove 7 2 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 10 librados: watch_flush enter 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 10 librados: watch_flush exit 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 10 client.4478.objecter close_session for osd.7 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7fc344081390 msgr2=0x7fc3440837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7fc344081390 0x7fc3440837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 20 client.4478.objecter put_session s=0x7fc344080cc0 osd=7 1 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 20 client.4478.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 10 client.4478.objecter successfully canceled tick 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc344078e50 msgr2=0x7fc34407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc344078e50 0x7fc34407b310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc36400a8d0 tx=0x7fc364008040 comp rx=0 tx=0).stop 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 10 monclient: shutdown 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 msgr2=0x7fc36c00f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 --2- 10.20.193.6:0/996581608 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc36c0e0fa0 0x7fc36c00f740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc370009f50 tx=0x7fc370004b40 comp rx=0 tx=0).stop 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37acf4640 1 -- 10.20.193.6:0/996581608 reap_dead start 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 shutdown_connections 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 >> 10.20.193.6:0/996581608 conn(0x7fc36c006e10 msgr2=0x7fc36c053fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 shutdown_connections 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 -- 10.20.193.6:0/996581608 wait complete. 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.786+0000 7fc37b4f5640 1 librados: shutdown 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.787+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== osd.15 v2:10.20.193.189:6826/3776650816 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7ff640083ea0 con 0x7ff640081390 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.787+0000 7ff65dffb640 10 client.4453.objecter ms_dispatch2 0x7ff670000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.787+0000 7ff65dffb640 10 client.4453.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.787+0000 7ff65dffb640 20 client.4453.objecter put_session s=0x7ff640080cc0 osd=15 4 2026-02-22T23:23:27.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.787+0000 7ff65dffb640 15 client.4453.objecter _session_command_op_remove 15 1 2026-02-22T23:23:27.799 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:27.799 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-22T23:23:27.799 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:27.800 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 10 client.4453.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 15 client.4453.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 20 client.4453.objecter _get_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 20 client.4453.objecter put_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 20 client.4453.objecter _get_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 15 client.4453.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 20 client.4453.objecter get_session s=0x7ff640080cc0 osd=15 3 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 15 client.4453.objecter _session_command_op_assign 15 2 2026-02-22T23:23:27.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 20 client.4453.objecter put_session s=0x7ff640080cc0 osd=15 4 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 10 client.4453.objecter _send_command 2 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.794+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 --> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff67015cce0 con 0x7ff640081390 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.796+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe0ec00aa00 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.796+0000 7fe0cffff640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe0c8078e50 0x7fe0c807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.796+0000 7fe0f67fc640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe0c8078e50 0x7fe0c807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.796+0000 7fe0f67fc640 10 monclient: get_auth_request con 0x7fe0c8078e50 auth_method 0 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.797+0000 7fe0f67fc640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe0c8078e50 0x7fe0c807b310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0f808e9e0 tx=0x7fe0e0031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.797+0000 7fe0cffff640 10 client.4456.objecter ms_handle_connect 0x7fe0c8078e50 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fe0ec00ad20 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 10 client.4456.objecter ms_dispatch2 0x7fe0f8000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 3 client.4456.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 3 client.4456.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 10 client.4456.objecter checking command 1 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fe0c80813f0 0x7fe0c8083830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 20 client.4456.objecter _get_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 20 client.4456.objecter put_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 15 client.4456.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 20 client.4456.objecter _get_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 20 client.4456.objecter get_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 15 client.4456.objecter _session_command_op_assign 13 1 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 20 client.4456.objecter put_session s=0x7fe0c8080d20 osd=13 4 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 10 client.4456.objecter _send_command 1 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 --> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0ec013e70 con 0x7fe0c80813f0 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 20 client.4456.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0ec05e490 con 0x7fe0f80d1ea0 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.798+0000 7fe0cffff640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.799+0000 7fe0f77fe640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fe0c80813f0 0x7fe0c8083830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.799+0000 7fe0f77fe640 10 monclient: get_auth_request con 0x7fe0c80813f0 auth_method 0 2026-02-22T23:23:27.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.801+0000 7fe0cffff640 10 client.4456.objecter ms_handle_connect 0x7fe0c80813f0 2026-02-22T23:23:27.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.801+0000 7fe0f77fe640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fe0c80813f0 0x7fe0c8083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.13 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff65dffb640 1 -- 10.20.193.6:0/1016129297 <== osd.15 v2:10.20.193.189:6826/3776650816 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff67015cce0 con 0x7ff640081390 2026-02-22T23:23:27.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff65dffb640 10 client.4453.objecter ms_dispatch2 0x7ff670000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff65dffb640 10 client.4453.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff65dffb640 20 client.4453.objecter put_session s=0x7ff640080cc0 osd=15 4 2026-02-22T23:23:27.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff65dffb640 15 client.4453.objecter _session_command_op_remove 15 2 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 10 librados: watch_flush enter 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 10 librados: watch_flush exit 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 10 client.4453.objecter close_session for osd.15 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7ff640081390 msgr2=0x7ff6400837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7ff640081390 0x7ff6400837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 20 client.4453.objecter put_session s=0x7ff640080cc0 osd=15 1 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 20 client.4453.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 10 client.4453.objecter successfully canceled tick 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff640078e50 msgr2=0x7ff64007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff640078e50 0x7ff64007b310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff660009380 tx=0x7ff660033000 comp rx=0 tx=0).stop 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 10 monclient: shutdown 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff67015d2c0 msgr2=0x7ff67014c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 --2- 10.20.193.6:0/1016129297 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff67015d2c0 0x7ff67014c240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff66400b580 tx=0x7ff66400ba50 comp rx=0 tx=0).stop 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff675b0e640 1 -- 10.20.193.6:0/1016129297 reap_dead start 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 shutdown_connections 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 >> 10.20.193.6:0/1016129297 conn(0x7ff6700ce730 msgr2=0x7ff67006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 shutdown_connections 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 -- 10.20.193.6:0/1016129297 wait complete. 2026-02-22T23:23:27.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.802+0000 7ff677598640 1 librados: shutdown 2026-02-22T23:23:27.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.805+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== osd.10 v2:10.20.193.80:6808/2974128537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f5f58083ea0 con 0x7f5f58081390 2026-02-22T23:23:27.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.805+0000 7f5f63fff640 10 client.4440.objecter ms_dispatch2 0x7f5f88000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.805+0000 7f5f63fff640 10 client.4440.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.805+0000 7f5f63fff640 20 client.4440.objecter put_session s=0x7f5f58080cc0 osd=10 4 2026-02-22T23:23:27.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.805+0000 7f5f63fff640 15 client.4440.objecter _session_command_op_remove 10 1 2026-02-22T23:23:27.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.806+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== osd.3 v2:10.20.193.189:6818/2996878330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f8838083f30 con 0x7f8838081420 2026-02-22T23:23:27.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.806+0000 7f88567fc640 10 client.4447.objecter ms_dispatch2 0x7f8868000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.806+0000 7f88567fc640 10 client.4447.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.806+0000 7f88567fc640 20 client.4447.objecter put_session s=0x7f8838080d50 osd=3 4 2026-02-22T23:23:27.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.806+0000 7f88567fc640 15 client.4447.objecter _session_command_op_remove 3 1 2026-02-22T23:23:27.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.806+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f65c803f050 con 0x7f65e015f650 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.806+0000 7f65d57fa640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f65b0078e00 0x7f65b007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7f65e4c86640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f65b0078e00 0x7f65b007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7f65e4c86640 10 monclient: get_auth_request con 0x7f65b0078e00 auth_method 0 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7f65d57fa640 10 client.4459.objecter ms_handle_connect 0x7f65b0078e00 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7f65e4c86640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f65b0078e00 0x7f65b007b2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65d000a830 tx=0x7f65d0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== osd.2 v2:10.20.193.80:6816/1223585399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd08802fa90 con 0x7fd06c0813c0 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7fd077fff640 10 client.4443.objecter ms_dispatch2 0x7fd09c000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7fd077fff640 10 client.4443.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7fd077fff640 20 client.4443.objecter put_session s=0x7fd06c080cf0 osd=2 4 2026-02-22T23:23:27.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.807+0000 7fd077fff640 15 client.4443.objecter _session_command_op_remove 2 1 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f65c80bc910 con 0x7f65e015f650 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 10 client.4459.objecter ms_dispatch2 0x7f65e0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 3 client.4459.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 3 client.4459.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 10 client.4459.objecter checking command 1 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f65b0081340 0x7f65b0083780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 20 client.4459.objecter _get_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 20 client.4459.objecter put_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 15 client.4459.objecter _session_command_op_remove -1 1 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 20 client.4459.objecter _get_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 20 client.4459.objecter get_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 15 client.4459.objecter _session_command_op_assign 5 1 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 20 client.4459.objecter put_session s=0x7f65b0080c70 osd=5 4 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 10 client.4459.objecter _send_command 1 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 --> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65c800af30 con 0x7f65b0081340 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 20 client.4459.objecter dump_active .. 0 homeless 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f65c8083430 con 0x7f65e015f650 2026-02-22T23:23:27.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65d57fa640 10 monclient: handle_get_version_reply finishing 1 version 27 2026-02-22T23:23:27.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.808+0000 7f65e5487640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f65b0081340 0x7f65b0083780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.809+0000 7f65e5487640 10 monclient: get_auth_request con 0x7f65b0081340 auth_method 0 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:27.812 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.810+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== osd.6 v2:10.20.193.80:6800/144861518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7ff5d0083ea0 con 0x7ff5d0081390 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.810+0000 7ff5ee7fc640 10 client.4450.objecter ms_dispatch2 0x7ff600000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.810+0000 7ff5ee7fc640 10 client.4450.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.810+0000 7ff5ee7fc640 20 client.4450.objecter put_session s=0x7ff5d0080cc0 osd=6 4 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.810+0000 7ff5ee7fc640 15 client.4450.objecter _session_command_op_remove 6 1 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.811+0000 7f65d57fa640 10 client.4459.objecter ms_handle_connect 0x7f65b0081340 2026-02-22T23:23:27.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.811+0000 7f65e5487640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f65b0081340 0x7f65b0083780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.812+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== osd.11 v2:10.20.193.189:6802/2070530833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fe5f8048ec0 con 0x7fe5dc081390 2026-02-22T23:23:27.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.812+0000 7fe5e7fff640 10 client.4425.objecter ms_dispatch2 0x7fe60c000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.812+0000 7fe5e7fff640 10 client.4425.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.812+0000 7fe5e7fff640 20 client.4425.objecter put_session s=0x7fe5dc080cc0 osd=11 4 2026-02-22T23:23:27.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.812+0000 7fe5e7fff640 15 client.4425.objecter _session_command_op_remove 11 1 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.813+0000 7f5f8e144640 10 client.4440.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.813+0000 7f5f8e144640 15 client.4440.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 20 client.4440.objecter _get_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 20 client.4440.objecter put_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 20 client.4440.objecter _get_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 15 client.4440.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 20 client.4440.objecter get_session s=0x7f5f58080cc0 osd=10 3 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 15 client.4440.objecter _session_command_op_assign 10 2 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 20 client.4440.objecter put_session s=0x7f5f58080cc0 osd=10 4 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 10 client.4440.objecter _send_command 2 2026-02-22T23:23:27.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.814+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 --> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5f88150990 con 0x7f5f58081390 2026-02-22T23:23:27.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 10 client.4443.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 15 client.4443.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 20 client.4443.objecter _get_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 20 client.4443.objecter put_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 20 client.4443.objecter _get_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 15 client.4443.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 20 client.4443.objecter get_session s=0x7fd06c080cf0 osd=2 3 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 15 client.4443.objecter _session_command_op_assign 2 2 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 20 client.4443.objecter put_session s=0x7fd06c080cf0 osd=2 4 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 10 client.4443.objecter _send_command 2 2026-02-22T23:23:27.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.815+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 --> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd09c159810 con 0x7fd06c0813c0 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 10 client.4447.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 15 client.4447.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 20 client.4447.objecter _get_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 20 client.4447.objecter put_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 20 client.4447.objecter _get_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 15 client.4447.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 20 client.4447.objecter get_session s=0x7f8838080d50 osd=3 3 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 15 client.4447.objecter _session_command_op_assign 3 2 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 20 client.4447.objecter put_session s=0x7f8838080d50 osd=3 4 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 10 client.4447.objecter _send_command 2 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.816+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f886814e790 con 0x7f8838081420 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.817+0000 7f5f63fff640 1 -- 10.20.193.6:0/1363421420 <== osd.10 v2:10.20.193.80:6808/2974128537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5f88150990 con 0x7f5f58081390 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.817+0000 7f5f63fff640 10 client.4440.objecter ms_dispatch2 0x7f5f88000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.817+0000 7f5f63fff640 10 client.4440.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.817+0000 7f5f63fff640 20 client.4440.objecter put_session s=0x7f5f58080cc0 osd=10 4 2026-02-22T23:23:27.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.817+0000 7f5f63fff640 15 client.4440.objecter _session_command_op_remove 10 2 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.817+0000 7f5f8e144640 10 librados: watch_flush enter 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 10 librados: watch_flush exit 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 10 client.4440.objecter close_session for osd.10 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7f5f58081390 msgr2=0x7f5f580837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7f5f58081390 0x7f5f580837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 20 client.4440.objecter put_session s=0x7f5f58080cc0 osd=10 1 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 20 client.4440.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 10 client.4440.objecter successfully canceled tick 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5f58078e50 msgr2=0x7f5f5807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5f58078e50 0x7f5f5807b310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006fd0 tx=0x7f5f74006d40 comp rx=0 tx=0).stop 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 10 monclient: shutdown 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5f881527f0 msgr2=0x7f5f881f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 --2- 10.20.193.6:0/1363421420 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5f881527f0 0x7f5f881f5a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c730 tx=0x7f5f7800cc00 comp rx=0 tx=0).stop 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f87fff640 1 -- 10.20.193.6:0/1363421420 reap_dead start 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 shutdown_connections 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 >> 10.20.193.6:0/1363421420 conn(0x7f5f880ce730 msgr2=0x7f5f8815ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 shutdown_connections 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 -- 10.20.193.6:0/1363421420 wait complete. 2026-02-22T23:23:27.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.818+0000 7f5f8e144640 1 librados: shutdown 2026-02-22T23:23:27.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd077fff640 1 -- 10.20.193.6:0/2662627114 <== osd.2 v2:10.20.193.80:6816/1223585399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd09c159810 con 0x7fd06c0813c0 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd077fff640 10 client.4443.objecter ms_dispatch2 0x7fd09c000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd077fff640 10 client.4443.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd077fff640 20 client.4443.objecter put_session s=0x7fd06c080cf0 osd=2 4 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd077fff640 15 client.4443.objecter _session_command_op_remove 2 2 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 10 librados: watch_flush enter 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 10 librados: watch_flush exit 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 10 client.4443.objecter close_session for osd.2 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd06c0813c0 msgr2=0x7fd06c083800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd06c0813c0 0x7fd06c083800 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 20 client.4443.objecter put_session s=0x7fd06c080cf0 osd=2 1 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 20 client.4443.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 10 client.4443.objecter successfully canceled tick 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd06c078e80 msgr2=0x7fd06c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd06c078e80 0x7fd06c07b340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0c8c40 tx=0x7fd08c006cf0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 10 monclient: shutdown 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c0ceca0 msgr2=0x7fd09c1f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 --2- 10.20.193.6:0/2662627114 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd09c0ceca0 0x7fd09c1f76a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd088002410 tx=0x7fd0880047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd09bfff640 1 -- 10.20.193.6:0/2662627114 reap_dead start 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 shutdown_connections 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.819+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 >> 10.20.193.6:0/2662627114 conn(0x7fd09c072fc0 msgr2=0x7fd09c0cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.820+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 shutdown_connections 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.820+0000 7fd0a1b5c640 1 -- 10.20.193.6:0/2662627114 wait complete. 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.820+0000 7fd0a1b5c640 1 librados: shutdown 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f88567fc640 1 -- 10.20.193.6:0/2575574876 <== osd.3 v2:10.20.193.189:6818/2996878330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f886814e790 con 0x7f8838081420 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f88567fc640 10 client.4447.objecter ms_dispatch2 0x7f8868000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f88567fc640 10 client.4447.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f88567fc640 20 client.4447.objecter put_session s=0x7f8838080d50 osd=3 4 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f88567fc640 15 client.4447.objecter _session_command_op_remove 3 2 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 10 librados: watch_flush enter 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 10 librados: watch_flush exit 2026-02-22T23:23:27.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 10 client.4447.objecter close_session for osd.3 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f8838081420 msgr2=0x7f8838083860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f8838081420 0x7f8838083860 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 20 client.4447.objecter put_session s=0x7f8838080d50 osd=3 1 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 20 client.4447.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 10 client.4447.objecter successfully canceled tick 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8838078e80 msgr2=0x7f883807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8838078e80 0x7f883807b340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8858009750 tx=0x7f8858006ca0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 10 monclient: shutdown 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f886815f650 msgr2=0x7f8868156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 --2- 10.20.193.6:0/2575574876 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f886815f650 0x7f8868156950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f885c00c700 tx=0x7f885c00cbd0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f886e6a2640 1 -- 10.20.193.6:0/2575574876 reap_dead start 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 shutdown_connections 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 >> 10.20.193.6:0/2575574876 conn(0x7f8868072fc0 msgr2=0x7f88680cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.821+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 shutdown_connections 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 10 client.4450.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 15 client.4450.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 20 client.4450.objecter _get_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 20 client.4450.objecter put_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 20 client.4450.objecter _get_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 15 client.4450.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 20 client.4450.objecter get_session s=0x7ff5d0080cc0 osd=6 3 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 15 client.4450.objecter _session_command_op_assign 6 2 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 20 client.4450.objecter put_session s=0x7ff5d0080cc0 osd=6 4 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 10 client.4450.objecter _send_command 2 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 --> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff6000d2b20 con 0x7ff5d0081390 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7f887012c640 1 -- 10.20.193.6:0/2575574876 wait complete. 2026-02-22T23:23:27.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.822+0000 7f887012c640 1 librados: shutdown 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 10 client.4425.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 15 client.4425.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 20 client.4425.objecter _get_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 20 client.4425.objecter put_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 20 client.4425.objecter _get_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 15 client.4425.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 20 client.4425.objecter get_session s=0x7fe5dc080cc0 osd=11 3 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 15 client.4425.objecter _session_command_op_assign 11 2 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 20 client.4425.objecter put_session s=0x7fe5dc080cc0 osd=11 4 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 10 client.4425.objecter _send_command 2 2026-02-22T23:23:27.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.823+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 --> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe60c0c67f0 con 0x7fe5dc081390 2026-02-22T23:23:27.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.825+0000 7ff5ee7fc640 1 -- 10.20.193.6:0/1494014871 <== osd.6 v2:10.20.193.80:6800/144861518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff6000d2b20 con 0x7ff5d0081390 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.825+0000 7ff5ee7fc640 10 client.4450.objecter ms_dispatch2 0x7ff600000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.825+0000 7ff5ee7fc640 10 client.4450.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.825+0000 7ff5ee7fc640 20 client.4450.objecter put_session s=0x7ff5d0080cc0 osd=6 4 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.825+0000 7ff5ee7fc640 15 client.4450.objecter _session_command_op_remove 6 2 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7ff607ccd640 10 librados: watch_flush enter 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7fe5e7fff640 1 -- 10.20.193.6:0/3918289228 <== osd.11 v2:10.20.193.189:6802/2070530833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe60c0c67f0 con 0x7fe5dc081390 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7fe5e7fff640 10 client.4425.objecter ms_dispatch2 0x7fe60c000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7fe5e7fff640 10 client.4425.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7fe5e7fff640 20 client.4425.objecter put_session s=0x7fe5dc080cc0 osd=11 4 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7fe5e7fff640 15 client.4425.objecter _session_command_op_remove 11 2 2026-02-22T23:23:27.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7fe611ed3640 10 librados: watch_flush enter 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.826+0000 7fe611ed3640 10 librados: watch_flush exit 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 10 librados: watch_flush exit 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 10 client.4450.objecter close_session for osd.6 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7ff5d0081390 msgr2=0x7ff5d00837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7ff5d0081390 0x7ff5d00837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 10 client.4425.objecter close_session for osd.11 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 20 client.4450.objecter put_session s=0x7ff5d0080cc0 osd=6 1 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 20 client.4450.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 10 client.4450.objecter successfully canceled tick 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fe5dc081390 msgr2=0x7fe5dc0837d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fe5dc081390 0x7fe5dc0837d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 20 client.4425.objecter put_session s=0x7fe5dc080cc0 osd=11 1 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 20 client.4425.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5d0078e50 msgr2=0x7ff5d007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5d0078e50 0x7ff5d007b310 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff6001ec490 tx=0x7ff5f4009110 comp rx=0 tx=0).stop 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 10 client.4425.objecter successfully canceled tick 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 10 monclient: shutdown 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff6000c9b00 msgr2=0x7ff6001f94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 --2- 10.20.193.6:0/1494014871 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff6000c9b00 0x7ff6001f94b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5f003a040 tx=0x7ff5f00047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff606243640 1 -- 10.20.193.6:0/1494014871 reap_dead start 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe5dc078e50 msgr2=0x7fe5dc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe5dc078e50 0x7fe5dc07b310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe60c15c600 tx=0x7fe5fc006d90 comp rx=0 tx=0).stop 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 shutdown_connections 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 >> 10.20.193.6:0/1494014871 conn(0x7ff600072fc0 msgr2=0x7ff6000cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 shutdown_connections 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 10 monclient: shutdown 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 -- 10.20.193.6:0/1494014871 wait complete. 2026-02-22T23:23:27.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c0c79e0 msgr2=0x7fe60c156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 --2- 10.20.193.6:0/3918289228 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe60c0c79e0 0x7fe60c156480 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8002410 tx=0x7fe5f80047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe60bfff640 1 -- 10.20.193.6:0/3918289228 reap_dead start 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7ff607ccd640 1 librados: shutdown 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 shutdown_connections 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 >> 10.20.193.6:0/3918289228 conn(0x7fe60c0cc190 msgr2=0x7fe60c14f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 shutdown_connections 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 -- 10.20.193.6:0/3918289228 wait complete. 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.827+0000 7fe611ed3640 1 librados: shutdown 2026-02-22T23:23:27.830 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:27.830 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-22T23:23:27.831 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:27.832 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:23:27.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== osd.5 v2:10.20.193.40:6800/2956321569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f65c800af30 con 0x7f65b0081340 2026-02-22T23:23:27.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7f65d57fa640 10 client.4459.objecter ms_dispatch2 0x7f65e0000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7f65d57fa640 10 client.4459.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7f65d57fa640 20 client.4459.objecter put_session s=0x7f65b0080c70 osd=5 4 2026-02-22T23:23:27.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7f65d57fa640 15 client.4459.objecter _session_command_op_remove 5 1 2026-02-22T23:23:27.835 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:27.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== osd.13 v2:10.20.193.40:6816/1944897191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fe0ec013e70 con 0x7fe0c80813f0 2026-02-22T23:23:27.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7fe0cffff640 10 client.4456.objecter ms_dispatch2 0x7fe0f8000b90 command_reply(tid 1: 0 ) 2026-02-22T23:23:27.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7fe0cffff640 10 client.4456.objecter _finish_command 1 = system:0 2026-02-22T23:23:27.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7fe0cffff640 20 client.4456.objecter put_session s=0x7fe0c8080d20 osd=13 4 2026-02-22T23:23:27.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.832+0000 7fe0cffff640 15 client.4456.objecter _session_command_op_remove 13 1 2026-02-22T23:23:27.835 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T23:23:27.838 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:27.839 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-22T23:23:27.839 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:27.839 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 10 client.4456.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 15 client.4456.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 20 client.4456.objecter _get_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 20 client.4456.objecter put_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 20 client.4456.objecter _get_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 15 client.4456.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 20 client.4456.objecter get_session s=0x7fe0c8080d20 osd=13 3 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 15 client.4456.objecter _session_command_op_assign 13 2 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 20 client.4456.objecter put_session s=0x7fe0c8080d20 osd=13 4 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 10 client.4456.objecter _send_command 2 2026-02-22T23:23:27.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 --> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0f8159810 con 0x7fe0c80813f0 2026-02-22T23:23:27.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 10 client.4459.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2026-02-22T23:23:27.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 15 client.4459.objecter _session_command_op_assign -1 2 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 20 client.4459.objecter _get_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 20 client.4459.objecter put_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 20 client.4459.objecter _get_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 15 client.4459.objecter _session_command_op_remove -1 2 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 20 client.4459.objecter get_session s=0x7f65b0080c70 osd=5 3 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 15 client.4459.objecter _session_command_op_assign 5 2 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 20 client.4459.objecter put_session s=0x7f65b0080c70 osd=5 4 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 10 client.4459.objecter _send_command 2 2026-02-22T23:23:27.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.840+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 --> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65e00d1960 con 0x7f65b0081340 2026-02-22T23:23:27.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.844+0000 7f65d57fa640 1 -- 10.20.193.6:0/391250990 <== osd.5 v2:10.20.193.40:6800/2956321569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65e00d1960 con 0x7f65b0081340 2026-02-22T23:23:27.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.844+0000 7f65d57fa640 10 client.4459.objecter ms_dispatch2 0x7f65e0000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.844+0000 7f65d57fa640 10 client.4459.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.844+0000 7f65d57fa640 20 client.4459.objecter put_session s=0x7f65b0080c70 osd=5 4 2026-02-22T23:23:27.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.844+0000 7f65d57fa640 15 client.4459.objecter _session_command_op_remove 5 2 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 10 librados: watch_flush enter 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 10 librados: watch_flush exit 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 10 client.4459.objecter close_session for osd.5 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f65b0081340 msgr2=0x7f65b0083780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f65b0081340 0x7f65b0083780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 20 client.4459.objecter put_session s=0x7f65b0080c70 osd=5 1 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 20 client.4459.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 10 client.4459.objecter successfully canceled tick 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f65b0078e00 msgr2=0x7f65b007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f65b0078e00 0x7f65b007b2c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f65d000a830 tx=0x7f65d0008040 comp rx=0 tx=0).stop 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 10 monclient: shutdown 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015f650 msgr2=0x7f65e00c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 --2- 10.20.193.6:0/391250990 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65e015f650 0x7f65e00c9dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65c802f370 tx=0x7f65c80047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e5487640 1 -- 10.20.193.6:0/391250990 reap_dead start 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 shutdown_connections 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 >> 10.20.193.6:0/391250990 conn(0x7f65e0072fc0 msgr2=0x7f65e00d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 shutdown_connections 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 -- 10.20.193.6:0/391250990 wait complete. 2026-02-22T23:23:27.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.845+0000 7f65e6f11640 1 librados: shutdown 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0cffff640 1 -- 10.20.193.6:0/3017583479 <== osd.13 v2:10.20.193.40:6816/1944897191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe0f8159810 con 0x7fe0c80813f0 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0cffff640 10 client.4456.objecter ms_dispatch2 0x7fe0f8000b90 command_reply(tid 2: 0 ) 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0cffff640 10 client.4456.objecter _finish_command 2 = system:0 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0cffff640 20 client.4456.objecter put_session s=0x7fe0c8080d20 osd=13 4 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0cffff640 15 client.4456.objecter _session_command_op_remove 13 2 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 10 librados: watch_flush enter 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 10 librados: watch_flush exit 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 10 client.4456.objecter close_session for osd.13 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fe0c80813f0 msgr2=0x7fe0c8083830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fe0c80813f0 0x7fe0c8083830 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 20 client.4456.objecter put_session s=0x7fe0c8080d20 osd=13 1 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 20 client.4456.objecter shutdown clearing up homeless session... 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 10 client.4456.objecter successfully canceled tick 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe0c8078e50 msgr2=0x7fe0c807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe0c8078e50 0x7fe0c807b310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0f808e9e0 tx=0x7fe0e0031000 comp rx=0 tx=0).stop 2026-02-22T23:23:27.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 10 monclient: shutdown 2026-02-22T23:23:27.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f80d1ea0 msgr2=0x7fe0f80d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0fda1d640 1 --2- 10.20.193.6:0/3017583479 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe0f80d1ea0 0x7fe0f80d0e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c700 tx=0x7fe0ec00cbd0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.846+0000 7fe0f77fe640 1 -- 10.20.193.6:0/3017583479 reap_dead start 2026-02-22T23:23:27.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.848+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 shutdown_connections 2026-02-22T23:23:27.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.848+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 >> 10.20.193.6:0/3017583479 conn(0x7fe0f8072fc0 msgr2=0x7fe0f814a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.848+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 shutdown_connections 2026-02-22T23:23:27.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.848+0000 7fe0fda1d640 1 -- 10.20.193.6:0/3017583479 wait complete. 2026-02-22T23:23:27.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.848+0000 7fe0fda1d640 1 librados: shutdown 2026-02-22T23:23:27.857 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:27.857 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-22T23:23:27.860 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:27.860 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-22T23:23:27.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.862+0000 7f58a1dd6640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.862+0000 7f58a1dd6640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.862+0000 7f58a1dd6640 10 monclient: monmap: 2026-02-22T23:23:27.863 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.863 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.863 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.863309+0000 2026-02-22T23:23:27.863 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.863309+0000 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.864 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 1 Processor -- start 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 1 -- start start 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient: init 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient: _add_conns ranks=[5,2,4,3,0,1] 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f589c155f80 0x7f589c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient(hunting): picked mon.noname-c con 0x7f589c155f80 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f589c0ccb40 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f589c15da30 0x7f589c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient(hunting): picked mon.noname-b con 0x7f589c15da30 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589c0d0380 con 0x7f589c0ccb40 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f589c06d5b0 con 0x7f589c15da30 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f589c06c9a0 con 0x7f589c155f80 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.863+0000 7f58a1dd6640 10 monclient(hunting): authenticate will time out at 1082.590967s 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589b7fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f589c155f80 0x7f589c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589bfff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f589c15da30 0x7f589c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589bfff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f589c15da30 0x7f589c0caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33234/0 (socket says 10.20.193.6:33234) 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589bfff640 1 -- 10.20.193.6:0/314655391 learned_addr learned my addr 10.20.193.6:0/314655391 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589bfff640 10 monclient(hunting): get_auth_request con 0x7f589c15da30 auth_method 0 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589b7fe640 10 monclient(hunting): get_auth_request con 0x7f589c155f80 auth_method 0 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.864+0000 7f589b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.865+0000 7f589bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.865+0000 7f589bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.865+0000 7f589bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.865+0000 7f589b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.865+0000 7f589b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.865+0000 7f589b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 10 monclient(hunting): handle_auth_done global_id 4462 payload 274 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 1 -- 10.20.193.6:0/314655391 >> v2:10.20.193.80:3300/0 conn(0x7f589c155f80 msgr2=0x7f589c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 1 --2- 10.20.193.6:0/314655391 >> v2:10.20.193.80:3300/0 conn(0x7f589c155f80 0x7f589c15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 1 -- 10.20.193.6:0/314655391 >> v1:10.20.193.80:6789/0 conn(0x7f589c0ccb40 legacy=0x7f589c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.866+0000 7f589bfff640 1 -- 10.20.193.6:0/314655391 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c06cc80 con 0x7f589c15da30 2026-02-22T23:23:27.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.867+0000 7f589bfff640 1 --2- 10.20.193.6:0/314655391 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c15da30 0x7f589c0caf30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f589c06f270 tx=0x7f589002f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c36c2c10d7f6a28 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 1 -- 10.20.193.6:0/314655391 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589002fcf0 con 0x7f589c15da30 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: dump: 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 1 -- 10.20.193.6:0/314655391 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58900373f0 con 0x7f589c15da30 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 1 -- 10.20.193.6:0/314655391 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58900377c0 con 0x7f589c15da30 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f589a7fc640 10 monclient: dump: 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 5 monclient: authenticate success, global_id 4462 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 4 set_mon_vals no callback set 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 10 monclient: shutdown 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 1 -- 10.20.193.6:0/314655391 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c15da30 msgr2=0x7f589c0caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/314655391 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c15da30 0x7f589c0caf30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f589c06f270 tx=0x7f589002f1f0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 1 -- 10.20.193.6:0/314655391 shutdown_connections 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/314655391 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c15da30 0x7f589c0caf30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/314655391 >> v2:10.20.193.80:3300/0 conn(0x7f589c155f80 0x7f589c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.869+0000 7f58a1dd6640 1 -- 10.20.193.6:0/314655391 >> 10.20.193.6:0/314655391 conn(0x7f589c0cc020 msgr2=0x7f589c0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.870+0000 7f58a1dd6640 1 -- 10.20.193.6:0/314655391 shutdown_connections 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.870+0000 7f58a1dd6640 1 -- 10.20.193.6:0/314655391 wait complete. 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 10 monclient: monmap: 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 1 librados: starting msgr at 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 1 librados: starting objecter 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 1 Processor -- start 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 1 -- start start 2026-02-22T23:23:27.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 1 librados: setting wanted keys 2026-02-22T23:23:27.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 1 librados: calling monclient init 2026-02-22T23:23:27.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.871+0000 7f58a1dd6640 10 monclient: init 2026-02-22T23:23:27.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.872+0000 7f58a1dd6640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.872+0000 7f58a1dd6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:27.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.872+0000 7f58a1dd6640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f589c0ccb40 0x7f589c1faae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.872+0000 7f58a1dd6640 10 monclient(hunting): picked mon.noname-c con 0x7f589c0ccb40 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.872+0000 7f58a1dd6640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c155f80 0x7f589c14b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.872+0000 7f58a1dd6640 10 monclient(hunting): picked mon.noname-b con 0x7f589c155f80 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f589c14bdb0 0x7f589c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f589b7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f589c0ccb40 0x7f589c1faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f589affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c155f80 0x7f589c14b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f589b7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f589c0ccb40 0x7f589c1faae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43478/0 (socket says 10.20.193.6:43478) 2026-02-22T23:23:27.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f589affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c155f80 0x7f589c14b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33236/0 (socket says 10.20.193.6:33236) 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 10 monclient(hunting): picked mon.noname-a con 0x7f589c14bdb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f589c072eb0 con 0x7f589c14bdb0 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f589c06d240 con 0x7f589c155f80 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f589c156360 con 0x7f589c0ccb40 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.873+0000 7f58a1dd6640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f58a1dd6640 10 monclient(hunting): authenticate will time out at 1082.601011s 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f589c14bdb0 0x7f589c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f589c14bdb0 0x7f589c070600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58840/0 (socket says 10.20.193.6:58840) 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 1 -- 10.20.193.6:0/2784236733 learned_addr learned my addr 10.20.193.6:0/2784236733 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 10 monclient(hunting): get_auth_request con 0x7f589c14bdb0 auth_method 0 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589b7fe640 10 monclient(hunting): get_auth_request con 0x7f589c0ccb40 auth_method 0 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589affd640 10 monclient(hunting): get_auth_request con 0x7f589c155f80 auth_method 0 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.874+0000 7f589affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.875+0000 7f589b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.875+0000 7f589b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.875+0000 7f589b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 10 monclient(hunting): handle_auth_done global_id 4449 payload 824 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 1 -- 10.20.193.6:0/2784236733 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f589c0ccb40 msgr2=0x7f589c1faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f589c0ccb40 0x7f589c1faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 1 -- 10.20.193.6:0/2784236733 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c155f80 msgr2=0x7f589c14b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c155f80 0x7f589c14b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 1 monclient: found mon.noname-a 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589bfff640 1 -- 10.20.193.6:0/2784236733 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c14c240 con 0x7f589c14bdb0 2026-02-22T23:23:27.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.876+0000 7f589b7fe640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f589c0ccb40 0x7f589c1faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:23:27.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.877+0000 7f589bfff640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f589c14bdb0 0x7f589c070600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5890039040 tx=0x7f5890004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.878+0000 7f5877fff640 10 client.?.objecter ms_handle_connect 0x7f589c14bdb0 2026-02-22T23:23:27.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.878+0000 7f5877fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.878+0000 7f5abb3eb640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.878+0000 7f5abb3eb640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.878+0000 7f5abb3eb640 10 monclient: monmap: 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.879462+0000 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.879462+0000 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.879+0000 7f5abb3eb640 1 Processor -- start 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.879+0000 7f5abb3eb640 1 -- start start 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.879+0000 7f5abb3eb640 10 monclient: init 2026-02-22T23:23:27.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.879+0000 7fd297103640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.879+0000 7fd297103640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.879+0000 7fd297103640 10 monclient: monmap: 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.880471+0000 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.880471+0000 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient: _add_conns ranks=[3,5,0,2,1,4] 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7fd297103640 1 Processor -- start 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7fd297103640 1 -- start start 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7fd297103640 10 monclient: init 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f5ab41527f0 0x7f5ab4073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient(hunting): picked mon.noname-a con 0x7f5ab41527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f5ab415d2c0 0x7f5ab415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient(hunting): picked mon.noname-c con 0x7f5ab415d2c0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f5ab415d730 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab4073c20 con 0x7f5ab415d730 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab4074b10 con 0x7f5ab41527f0 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab4073f00 con 0x7f5ab415d2c0 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5abb3eb640 10 monclient(hunting): authenticate will time out at 1082.607616s 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 1 -- 10.20.193.6:0/2784236733 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890046020 con 0x7f589c14bdb0 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 10 monclient: dump: 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f5877fff640 1 -- 10.20.193.6:0/2784236733 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5890041660 con 0x7f589c14bdb0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 5 monclient: authenticate success, global_id 4449 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 10 monclient: _renew_subs 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c1fc2e0 con 0x7f589c14bdb0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 10 client.4449.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 10 monclient: _renew_subs 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c06f560 con 0x7f589c14bdb0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f58a1dd6640 1 librados: init done 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f463712c640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f463712c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.880+0000 7f463712c640 10 monclient: monmap: 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.881516+0000 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.881516+0000 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5877fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5877fff640 1 -- 10.20.193.6:0/2784236733 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890041a70 con 0x7f589c14bdb0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5877fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5877fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:27.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5877fff640 10 monclient: dump: 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab895f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f5ab415d2c0 0x7f5ab415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5899ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient: _add_conns ranks=[0,2,5,1,4,3] 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab9961640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f5ab415d730 0x7f5ab40ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58804/0 (socket says 10.20.193.6:58804) 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab9961640 1 -- 10.20.193.6:0/4105684183 learned_addr learned my addr 10.20.193.6:0/4105684183 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab9160640 1 --2- 10.20.193.6:0/4105684183 >> v2:10.20.193.6:3300/0 conn(0x7f5ab41527f0 0x7f5ab4073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd29015f650 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd29015fa30 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd2900c9b00 0x7fd2900c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient(hunting): picked mon.noname-c con 0x7fd2900c9b00 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2900ca4b0 con 0x7fd29015f650 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f463712c640 1 Processor -- start 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd287fff640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fd29015fa30 0x7fd2900d1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:48174/0 (socket says 10.20.193.6:48174) 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd287fff640 1 -- 10.20.193.6:0/277600318 learned_addr learned my addr 10.20.193.6:0/277600318 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 1 -- 10.20.193.6:0/277600318 --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2900d2450 con 0x7fd29015fa30 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 1 -- 10.20.193.6:0/277600318 --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd2900c8dd0 con 0x7fd2900c9b00 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7fd297103640 10 monclient(hunting): authenticate will time out at 1082.608851s 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f463712c640 1 -- start start 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab9160640 10 monclient(hunting): get_auth_request con 0x7f5ab41527f0 auth_method 0 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab9160640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab9160640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f5ab9160640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f463712c640 10 monclient: init 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f58a1dd6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f58a1dd6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f58a1dd6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.881+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c072bd0 con 0x7f589c14bdb0 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab895f640 10 monclient(hunting): get_auth_request con 0x7f5ab415d2c0 auth_method 0 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab895f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab895f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab895f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fd295679640 1 --2- 10.20.193.6:0/277600318 >> v2:10.20.193.80:3300/0 conn(0x7fd2900c9b00 0x7fd2900c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab9160640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab9160640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab9160640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab895f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab895f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient: _add_conns ranks=[3,1,4,0,5,2] 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f46301527f0 0x7f4630073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f5ab895f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient(hunting): picked mon.noname-a con 0x7f46301527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f463015d2c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f463015d730 0x7f46300ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient(hunting): picked mon.noname-b con 0x7f463015d730 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4630073c20 con 0x7f463015d2c0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4630074b10 con 0x7f46301527f0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f4630073f00 con 0x7f463015d730 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fd295679640 10 monclient(hunting): get_auth_request con 0x7fd2900c9b00 auth_method 0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f463712c640 10 monclient(hunting): authenticate will time out at 1082.609553s 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fd295679640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fd295679640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fd295679640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7f46356a2640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f463015d730 0x7f46300ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fad28c61640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fad28c61640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.882+0000 7fad28c61640 10 monclient: monmap: 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.887 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.883517+0000 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.883517+0000 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f463015d730 0x7f46300ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33252/0 (socket says 10.20.193.6:33252) 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 1 -- 10.20.193.6:0/2768445442 learned_addr learned my addr 10.20.193.6:0/2768445442 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 10 monclient(hunting): get_auth_request con 0x7f463015d730 auth_method 0 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 10 monclient(hunting): handle_auth_done global_id 4496 payload 274 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 1 -- 10.20.193.6:0/4105684183 >> v2:10.20.193.6:3300/0 conn(0x7f5ab41527f0 msgr2=0x7f5ab4073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 1 --2- 10.20.193.6:0/4105684183 >> v2:10.20.193.6:3300/0 conn(0x7f5ab41527f0 0x7f5ab4073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 1 -- 10.20.193.6:0/4105684183 >> v1:10.20.193.6:6789/0 conn(0x7f5ab415d730 legacy=0x7f5ab40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7fad28c61640 1 Processor -- start 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f5ab895f640 1 -- 10.20.193.6:0/4105684183 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab40741e0 con 0x7f5ab415d2c0 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7fad28c61640 1 -- start start 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7fad28c61640 10 monclient: init 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7f46356a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7fd295679640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7fd295679640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.883+0000 7fd295679640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient: _add_conns ranks=[0,1,3,2,5,4] 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fad241527f0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fad24073530 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fad2415fad0 0x7fad240ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient(hunting): picked mon.noname-a con 0x7fad2415fad0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad240739a0 con 0x7fad241527f0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad2415d9d0 con 0x7fad24073530 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fad24074b10 con 0x7fad2415fad0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7fad28c61640 10 monclient(hunting): authenticate will time out at 1082.611262s 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 10 monclient(hunting): handle_auth_done global_id 4468 payload 274 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 1 -- 10.20.193.6:0/2768445442 >> v2:10.20.193.6:3300/0 conn(0x7f46301527f0 msgr2=0x7f4630073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 1 --2- 10.20.193.6:0/2768445442 >> v2:10.20.193.6:3300/0 conn(0x7f46301527f0 0x7f4630073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 1 -- 10.20.193.6:0/2768445442 >> v1:10.20.193.40:6789/0 conn(0x7f463015d2c0 legacy=0x7f463015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.884+0000 7f46356a2640 1 -- 10.20.193.6:0/2768445442 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46300741e0 con 0x7f463015d730 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 10 monclient(hunting): handle_auth_done global_id 4499 payload 274 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 1 -- 10.20.193.6:0/277600318 >> v1:10.20.193.80:6789/0 conn(0x7fd29015fa30 legacy=0x7fd2900d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 1 -- 10.20.193.6:0/277600318 >> v1:10.20.193.6:6789/0 conn(0x7fd29015f650 legacy=0x7fd2900ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fd295679640 1 -- 10.20.193.6:0/277600318 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2900d2730 con 0x7fd2900c9b00 2026-02-22T23:23:27.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7f4acbde4640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7f4acbde4640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7f4acbde4640 10 monclient: monmap: 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.886404+0000 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.886404+0000 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.893 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.893 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.893 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.893 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7f99f68b0640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7f99f68b0640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7f99f68b0640 10 monclient: monmap: 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.886482+0000 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.886482+0000 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fad21d74640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fad24073530 0x7fad2415f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:53082/0 (socket says 10.20.193.6:53082) 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.885+0000 7fad21d74640 1 -- 10.20.193.6:0/1347071558 learned_addr learned my addr 10.20.193.6:0/1347071558 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f4acbde4640 1 Processor -- start 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f4acbde4640 1 -- start start 2026-02-22T23:23:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f4acbde4640 10 monclient: init 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f5ab895f640 1 --2- 10.20.193.6:0/4105684183 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d2c0 0x7f5ab415f3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5ab408f510 tx=0x7f5aa802ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc4938192514b7d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f99f68b0640 1 Processor -- start 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f99f68b0640 1 -- start start 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f99f68b0640 10 monclient: init 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f4acbde4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.886+0000 7f99f68b0640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 10 monclient: _add_conns ranks=[1,0,2,4,3,5] 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4ac4153ed0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4ac40c7950 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f770fafc640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f770fafc640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f770fafc640 10 monclient: monmap: 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.887679+0000 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.887679+0000 2026-02-22T23:23:27.895 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 10 monclient: _add_conns ranks=[1,0,4,5,3,2] 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4ac40ccb40 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac40d0380 con 0x7f4ac40c7950 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f99f01527f0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f99f0073530 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac40c9270 con 0x7f4ac4153ed0 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac40c9550 con 0x7f4ac40ccb40 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f46356a2640 1 --2- 10.20.193.6:0/2768445442 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d730 0x7f46300ccf70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f463006cdf0 tx=0x7f462c02f2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b047756905bf6d5c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f4acbde4640 10 monclient(hunting): authenticate will time out at 1082.614250s 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f99f015fad0 0x7f99f00ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 10 monclient(hunting): picked mon.noname-b con 0x7f99f015fad0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f00739a0 con 0x7f99f0073530 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f015d9d0 con 0x7f99f01527f0 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f99f0074b10 con 0x7f99f015fad0 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7fd295679640 1 --2- 10.20.193.6:0/277600318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd2900c9b00 0x7fd2900c9f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d90 tx=0x7fd28c02f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d29550c947aa983 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f99f68b0640 10 monclient(hunting): authenticate will time out at 1082.614098s 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f770fafc640 1 Processor -- start 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f770fafc640 1 -- start start 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.887+0000 7f770fafc640 10 monclient: init 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient: _add_conns ranks=[5,3,2,4,0,1] 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f770815f650 0x7f770815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient(hunting): picked mon.noname-c con 0x7f770815f650 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f77080ceca0 0x7f77080d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient(hunting): picked mon.noname-a con 0x7f77080ceca0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f99f4e26640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f99f015fad0 0x7f99f00ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f77080d1ea0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77080d2450 con 0x7f77080d1ea0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f77080c8dd0 con 0x7f77080ceca0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f77080d2730 con 0x7f770815f650 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.888+0000 7f770fafc640 10 monclient(hunting): authenticate will time out at 1082.615614s 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f4ac9b59640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f4ac4153ed0 0x7f4ac40c7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:53088/0 (socket says 10.20.193.6:53088) 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f4ac9b59640 1 -- 10.20.193.6:0/3178618509 learned_addr learned my addr 10.20.193.6:0/3178618509 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 1 -- 10.20.193.6:0/4105684183 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa803c070 con 0x7f5ab415d2c0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: dump: 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.898 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 1 -- 10.20.193.6:0/4105684183 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa802faf0 con 0x7f5ab415d2c0 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 1 -- 10.20.193.6:0/4105684183 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8037440 con 0x7f5ab415d2c0 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5abb3eb640 5 monclient: authenticate success, global_id 4496 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5abb3eb640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5abb3eb640 4 set_mon_vals no callback set 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f5aa3fff640 10 monclient: dump: 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f99f4e26640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f99f015fad0 0x7f99f00ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33268/0 (socket says 10.20.193.6:33268) 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f99f4e26640 1 -- 10.20.193.6:0/1333614327 learned_addr learned my addr 10.20.193.6:0/1333614327 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f99f4e26640 10 monclient(hunting): get_auth_request con 0x7f99f015fad0 auth_method 0 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f99f4e26640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f99f4e26640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.889+0000 7f99f4e26640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 10 monclient: shutdown 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 1 -- 10.20.193.6:0/4105684183 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d2c0 msgr2=0x7f5ab415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/4105684183 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d2c0 0x7f5ab415f3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5ab408f510 tx=0x7f5aa802ede0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f770d871640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f770815f650 0x7f770815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 1 -- 10.20.193.6:0/4105684183 shutdown_connections 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/4105684183 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d2c0 0x7f5ab415f3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/4105684183 >> v2:10.20.193.6:3300/0 conn(0x7f5ab41527f0 0x7f5ab4073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 1 -- 10.20.193.6:0/4105684183 >> 10.20.193.6:0/4105684183 conn(0x7f5ab40ce730 msgr2=0x7f5ab40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f5abb3eb640 1 -- 10.20.193.6:0/4105684183 shutdown_connections 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f99f4e26640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f99f4e26640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7f99f4e26640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2905529005 0 0) 0x7fad2415d9d0 con 0x7fad24073530 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 1 -- 10.20.193.6:0/277600318 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c03c070 con 0x7fd2900c9b00 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: dump: 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.900 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fad21573640 10 monclient(hunting): _negotiate 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fad21573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 1 -- 10.20.193.6:0/277600318 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c02fe10 con 0x7fd2900c9b00 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd297103640 5 monclient: authenticate success, global_id 4499 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd297103640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fad21573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fad21573640 10 monclient(hunting): my global_id is 4474 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd297103640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 1 -- 10.20.193.6:0/277600318 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c037660 con 0x7fd2900c9b00 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd297103640 4 set_mon_vals no callback set 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fd2877fe640 10 monclient: dump: 2026-02-22T23:23:27.901 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.890+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad10003620 con 0x7fad24073530 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f770d871640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f770815f650 0x7f770815fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43508/0 (socket says 10.20.193.6:43508) 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f770d871640 1 -- 10.20.193.6:0/1436965060 learned_addr learned my addr 10.20.193.6:0/1436965060 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f770d871640 10 monclient(hunting): get_auth_request con 0x7f770815f650 auth_method 0 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f770d871640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f770d871640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f770d871640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f5abb3eb640 1 -- 10.20.193.6:0/4105684183 wait complete. 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 10 monclient: shutdown 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 1 -- 10.20.193.6:0/277600318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd2900c9b00 msgr2=0x7fd2900c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 1 --2- 10.20.193.6:0/277600318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd2900c9b00 0x7fd2900c9f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d90 tx=0x7fd28c02f150 comp rx=0 tx=0).stop 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 1 -- 10.20.193.6:0/277600318 shutdown_connections 2026-02-22T23:23:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 1 --2- 10.20.193.6:0/277600318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd2900c9b00 0x7fd2900c9f70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f5abb3eb640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7f5abb3eb640 10 monclient: monmap: 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 1 -- 10.20.193.6:0/277600318 >> 10.20.193.6:0/277600318 conn(0x7fd290072fc0 msgr2=0x7fd2900733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 1 -- 10.20.193.6:0/277600318 shutdown_connections 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 librados: starting msgr at 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 librados: starting objecter 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 Processor -- start 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 10 monclient(hunting): handle_auth_done global_id 4477 payload 274 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 1 -- 10.20.193.6:0/1333614327 >> v1:10.20.193.40:6789/0 conn(0x7f99f01527f0 legacy=0x7f99f015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 1 -- 10.20.193.6:0/1333614327 >> v1:10.20.193.6:6789/0 conn(0x7f99f0073530 legacy=0x7f99f015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f99f4e26640 1 -- 10.20.193.6:0/1333614327 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f0073c20 con 0x7f99f015fad0 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.891+0000 7fd297103640 1 -- 10.20.193.6:0/277600318 wait complete. 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7fd297103640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7fd297103640 10 monclient: monmap: 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7fd297103640 1 librados: starting msgr at 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7fd297103640 1 librados: starting objecter 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 Processor -- start 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 -- start start 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 librados: setting wanted keys 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 librados: calling monclient init 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient: init 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5ab41527f0 0x7f5ab415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): picked mon.noname-a con 0x7f5ab41527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5ab415d2c0 0x7f5ab42042e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): picked mon.noname-b con 0x7f5ab415d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d730 0x7f5ab415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): picked mon.noname-c con 0x7f5ab415d730 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4074fa0 con 0x7f5ab41527f0 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab40747a0 con 0x7f5ab415d2c0 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4075120 con 0x7f5ab415d730 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f5abb3eb640 10 monclient(hunting): authenticate will time out at 1082.619952s 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 1 -- 10.20.193.6:0/2768445442 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c02fda0 con 0x7f463015d730 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: dump: 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 1 -- 10.20.193.6:0/2768445442 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c0373f0 con 0x7f463015d730 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 1 -- 10.20.193.6:0/2768445442 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c037800 con 0x7f463015d730 2026-02-22T23:23:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f46277fe640 10 monclient: dump: 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f463712c640 5 monclient: authenticate success, global_id 4468 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f463712c640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.892+0000 7f463712c640 4 set_mon_vals no callback set 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 1 --2- 10.20.193.6:0/1347071558 >> v2:10.20.193.6:3300/0 conn(0x7fad2415fad0 0x7fad240ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 -- start start 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 librados: setting wanted keys 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 librados: calling monclient init 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient: init 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 10 monclient(hunting): get_auth_request con 0x7fad2415fad0 auth_method 0 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 10 monclient: shutdown 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 -- 10.20.193.6:0/2768445442 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d730 msgr2=0x7f46300ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 --2- 10.20.193.6:0/2768445442 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d730 0x7f46300ccf70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f463006cdf0 tx=0x7f462c02f2a0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 -- 10.20.193.6:0/2768445442 shutdown_connections 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 --2- 10.20.193.6:0/2768445442 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d730 0x7f46300ccf70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 --2- 10.20.193.6:0/2768445442 >> v2:10.20.193.6:3300/0 conn(0x7f46301527f0 0x7f4630073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 -- 10.20.193.6:0/2768445442 >> 10.20.193.6:0/2768445442 conn(0x7f46300ce730 msgr2=0x7f46300ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 -- 10.20.193.6:0/2768445442 shutdown_connections 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7f463712c640 1 -- 10.20.193.6:0/2768445442 wait complete. 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fad22d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd2900c9b00 0x7fd290150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): picked mon.noname-a con 0x7fd2900c9b00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd29015f650 0x7fd290201d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): picked mon.noname-c con 0x7fd29015f650 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd29015fa30 0x7fd290158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): picked mon.noname-b con 0x7fd29015fa30 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900c95e0 con 0x7fd2900c9b00 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900c9300 con 0x7fd29015fa30 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd2900d2070 con 0x7fd29015f650 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.893+0000 7fd297103640 10 monclient(hunting): authenticate will time out at 1082.620783s 2026-02-22T23:23:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f5ab9961640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d730 0x7f5ab415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1774673666 0 0) 0x7f4ac40c9550 con 0x7f4ac40ccb40 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f4ac8b57640 10 monclient(hunting): _negotiate 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f4ac8b57640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f4ac8b57640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f4ac8b57640 10 monclient(hunting): my global_id is 4505 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aac003620 con 0x7f4ac40ccb40 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient: monmap: 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 librados: starting msgr at 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 librados: starting objecter 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f770d871640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f770d871640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f770d871640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f5ab9961640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d730 0x7f5ab415a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43516/0 (socket says 10.20.193.6:43516) 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f5ab9961640 1 -- 10.20.193.6:0/3039883624 learned_addr learned my addr 10.20.193.6:0/3039883624 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f5ab9961640 10 monclient(hunting): get_auth_request con 0x7f5ab415d730 auth_method 0 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f5ab9961640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f5ab9961640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f5ab9961640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 Processor -- start 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 -- start start 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 librados: setting wanted keys 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 librados: calling monclient init 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient: init 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f46301527f0 0x7f46301f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): picked mon.noname-a con 0x7f46301527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d2c0 0x7f463015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): picked mon.noname-b con 0x7f463015d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f463015d730 0x7f4630148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): picked mon.noname-c con 0x7f463015d730 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4630074fa0 con 0x7f46301527f0 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f46300747a0 con 0x7f463015d2c0 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4630075120 con 0x7f463015d730 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.894+0000 7f463712c640 10 monclient(hunting): authenticate will time out at 1082.621919s 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f5ab9961640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f5ab9961640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f5ab9961640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 422505027 0 0) 0x7f4ac40c9270 con 0x7f4ac4153ed0 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f4ac8b57640 10 monclient(hunting): _negotiate 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f4ac8b57640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f4ac8b57640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f4ac8b57640 10 monclient(hunting): my global_id is 4480 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ac40c9550 con 0x7f4ac4153ed0 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 10 monclient(hunting): handle_auth_done global_id 4511 payload 274 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 1 -- 10.20.193.6:0/1436965060 >> v2:10.20.193.6:3300/0 conn(0x7f77080ceca0 msgr2=0x7f77080d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 1 --2- 10.20.193.6:0/1436965060 >> v2:10.20.193.6:3300/0 conn(0x7f77080ceca0 0x7f77080d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 1 -- 10.20.193.6:0/1436965060 >> v1:10.20.193.80:6789/0 conn(0x7f77080d1ea0 legacy=0x7f77080c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.896+0000 7f770d871640 1 -- 10.20.193.6:0/1436965060 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77080c9020 con 0x7f770815f650 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1276127272 0 0) 0x7f4aac003620 con 0x7f4ac40ccb40 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 >> v1:10.20.193.40:6789/0 conn(0x7f4ac4153ed0 legacy=0x7f4ac40c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 >> v1:10.20.193.6:6789/0 conn(0x7f4ac40c7950 legacy=0x7f4ac40caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac40c9830 con 0x7f4ac40ccb40 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4ac8b57640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4acbde4640 5 monclient: authenticate success, global_id 4505 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f4acbde4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.897+0000 7f99f4e26640 1 --2- 10.20.193.6:0/1333614327 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 0x7f99f00ccf70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009e50 tx=0x7f99e002f2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59bb587bce706a7c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 10 monclient(hunting): handle_auth_done global_id 4514 payload 824 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 1 -- 10.20.193.6:0/3039883624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5ab415d2c0 msgr2=0x7f5ab42042e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5ab415d2c0 0x7f5ab42042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 1 -- 10.20.193.6:0/3039883624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5ab41527f0 msgr2=0x7f5ab415c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5ab41527f0 0x7f5ab415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 1 monclient: found mon.noname-c 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f5ab9961640 1 -- 10.20.193.6:0/3039883624 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab415b090 con 0x7f5ab415d730 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2572458319 0 0) 0x7fad10003620 con 0x7fad24073530 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 >> v2:10.20.193.6:3300/0 conn(0x7fad2415fad0 msgr2=0x7fad240ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 --2- 10.20.193.6:0/1347071558 >> v2:10.20.193.6:3300/0 conn(0x7fad2415fad0 0x7fad240ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 >> v1:10.20.193.6:6789/0 conn(0x7fad241527f0 legacy=0x7fad2415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad24073c20 con 0x7fad24073530 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fad0c002f00 con 0x7fad24073530 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: dump: 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 >> v1:10.20.193.40:6789/0 conn(0x7fad24073530 legacy=0x7fad2415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10003ba0 0x7fad10003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): picked mon.a con 0x7fad10003ba0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad10004900 0x7fad10004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): picked mon.c con 0x7fad10004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad100053e0 0x7fad1000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): picked mon.b con 0x7fad100053e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad24074b10 con 0x7fad10003ba0 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad1000c2e0 con 0x7fad100053e0 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad1000c4e0 con 0x7fad10004900 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fad21573640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f4ac0003240 con 0x7f4ac40ccb40 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient: dump: 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 >> v1:10.20.193.80:6789/0 conn(0x7f4ac40ccb40 legacy=0x7f4ac40cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac003d30 0x7f4aac004110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): picked mon.a con 0x7f4aac003d30 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7fd295679640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd29015fa30 0x7fd290158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4aac004900 0x7f4aac004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): picked mon.c con 0x7f4aac004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4aac0054c0 0x7f4aac00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): picked mon.b con 0x7f4aac0054c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac00c370 con 0x7f4aac003d30 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac00c520 con 0x7f4aac0054c0 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac00c720 con 0x7f4aac004900 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4ac8b57640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.898+0000 7f4acbde4640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd29015fa30 0x7fd290158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33292/0 (socket says 10.20.193.6:33292) 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 1 -- 10.20.193.6:0/4130439526 learned_addr learned my addr 10.20.193.6:0/4130439526 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 10 monclient(hunting): get_auth_request con 0x7fd29015fa30 auth_method 0 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.899+0000 7fd295679640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.900+0000 7f5ab9961640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d730 0x7f5ab415a650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000cad0 tx=0x7f5ab000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7f5aa1ffb640 10 client.?.objecter ms_handle_connect 0x7f5ab415d730 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7f5aa1ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7fb28102b640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7fb28102b640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7fb28102b640 10 monclient: monmap: 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.901781+0000 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.901781+0000 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7fb28102b640 1 Processor -- start 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7fb28102b640 1 -- start start 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.901+0000 7fb28102b640 10 monclient: init 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient: _add_conns ranks=[3,1,2,5,0,4] 2026-02-22T23:23:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fb27c06ff20 0x7fb27c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient(hunting): picked mon.noname-a con 0x7fb27c06ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb27c0c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb27c15a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27c0c7e90 con 0x7fb27c0c99a0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27c0c8170 con 0x7fb27c15a5e0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb27c159a40 con 0x7fb27c06ff20 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 10 monclient(hunting): handle_auth_done global_id 4483 payload 824 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 1 -- 10.20.193.6:0/4130439526 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd29015f650 msgr2=0x7fd290201d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd29015f650 0x7fd290201d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 1 -- 10.20.193.6:0/4130439526 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd2900c9b00 msgr2=0x7fd290150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd2900c9b00 0x7fd290150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fd295679640 1 -- 10.20.193.6:0/4130439526 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd290151270 con 0x7fd29015fa30 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fb28102b640 10 monclient(hunting): authenticate will time out at 1082.629567s 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.902+0000 7fad21d74640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad10004900 0x7fad10004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.903+0000 7fad21d74640 10 monclient(hunting): get_auth_request con 0x7fad10004900 auth_method 0 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.903+0000 7fad21d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.903+0000 7fad21d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.903+0000 7fad21d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.903+0000 7fad21d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.903+0000 7fad21d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.903+0000 7fad21d74640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.904+0000 7fd295679640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd29015fa30 0x7fd290158b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd28c030fd0 tx=0x7fd28c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.904+0000 7fd2857fa640 10 client.?.objecter ms_handle_connect 0x7fd29015fa30 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.904+0000 7fd2857fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.904+0000 7f770d871640 1 --2- 10.20.193.6:0/1436965060 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f770815f650 0x7f770815fa30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f770808e9e0 tx=0x7f76f802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4182cdbc5f17f623 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.904+0000 7f4627fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d2c0 0x7f463015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 10 monclient(hunting): handle_auth_done global_id 4474 payload 293 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7f4627fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d2c0 0x7f463015fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:33304/0 (socket says 10.20.193.6:33304) 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 1 -- 10.20.193.6:0/1347071558 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad100053e0 msgr2=0x7fad1000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad100053e0 0x7fad1000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 1 -- 10.20.193.6:0/1347071558 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10003ba0 msgr2=0x7fad10003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10003ba0 0x7fad10003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 1 monclient: found mon.c 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 1 -- 10.20.193.6:0/1347071558 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad10003620 con 0x7fad10004900 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad21d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad28c61640 5 monclient: authenticate success, global_id 4474 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad28c61640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7fad22d76640 1 -- 10.20.193.6:0/1347071558 reap_dead start 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7f4627fff640 1 -- 10.20.193.6:0/1868574122 learned_addr learned my addr 10.20.193.6:0/1868574122 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7f4627fff640 10 monclient(hunting): get_auth_request con 0x7f463015d2c0 auth_method 0 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7f4627fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7f4627fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7f4627fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.905+0000 7f4ac9358640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4aac004900 0x7f4aac004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.906+0000 7fad21d74640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad10004900 0x7fad10004d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c270 tx=0x7fad0c02f5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edbb1d1377fb2820 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.906+0000 7f4ac9358640 10 monclient(hunting): get_auth_request con 0x7f4aac004900 auth_method 0 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.906+0000 7f4ac9358640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.906+0000 7f4ac9358640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.906+0000 7f4ac9358640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f4627fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f4627fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f4627fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f4aca35a640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4aac0054c0 0x7f4aac00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f4ac9358640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f4ac9358640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f4ac9358640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.907+0000 7f46356a2640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f463015d730 0x7f4630148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f46356a2640 10 monclient(hunting): get_auth_request con 0x7f463015d730 auth_method 0 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f46356a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f46356a2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f46356a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 10 monclient(hunting): handle_auth_done global_id 4486 payload 824 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 1 -- 10.20.193.6:0/1868574122 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f463015d730 msgr2=0x7f4630148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f463015d730 0x7f4630148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 1 -- 10.20.193.6:0/1868574122 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f46301527f0 msgr2=0x7f46301f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f46301527f0 0x7f46301f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4aca35a640 10 monclient(hunting): get_auth_request con 0x7f4aac0054c0 auth_method 0 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4aca35a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4aca35a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4aca35a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 1 monclient: found mon.noname-b 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.908+0000 7f4627fff640 1 -- 10.20.193.6:0/1868574122 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46301f1c20 con 0x7f463015d2c0 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 10 monclient(hunting): handle_auth_done global_id 4505 payload 293 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 1 -- 10.20.193.6:0/3178618509 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4aac0054c0 msgr2=0x7f4aac00bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4aac0054c0 0x7f4aac00bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 1 -- 10.20.193.6:0/3178618509 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac003d30 msgr2=0x7f4aac004110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac003d30 0x7f4aac004110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 10 monclient: _finish_hunting 0 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 1 monclient: found mon.c 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 1 -- 10.20.193.6:0/3178618509 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac003620 con 0x7f4aac004900 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4ac9358640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7fb27a575640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fb27c0c99a0 0x7fb27c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:53104/0 (socket says 10.20.193.6:53104) 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7fb27a575640 1 -- 10.20.193.6:0/3374040633 learned_addr learned my addr 10.20.193.6:0/3374040633 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:27.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f4aca35a640 1 -- 10.20.193.6:0/3178618509 reap_dead start 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 1 -- 10.20.193.6:0/3039883624 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0017070 con 0x7f5ab415d730 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 10 monclient: dump: 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5aa1ffb640 1 -- 10.20.193.6:0/3039883624 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab000ac70 con 0x7f5ab415d730 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.909+0000 7f5abb3eb640 5 monclient: authenticate success, global_id 4514 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5aa1ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5aa1ffb640 1 -- 10.20.193.6:0/3039883624 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0012640 con 0x7f5ab415d730 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5aa1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5aa37fe640 4 set_mon_vals no callback set 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5aa1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5aa1ffb640 10 monclient: dump: 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 monclient: _renew_subs 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab40c69b0 con 0x7f5ab415d730 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 client.4514.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 monclient: _renew_subs 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab40c6f70 con 0x7f5ab415d730 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 1 librados: init done 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.910+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab4152bd0 con 0x7f5ab415d730 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.911+0000 7f4627fff640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d2c0 0x7f463015fea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f461800c5b0 tx=0x7f461800ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.911+0000 7f46257fa640 10 client.?.objecter ms_handle_connect 0x7f463015d2c0 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.911+0000 7f46257fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.911+0000 7f4ac9358640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4aac004900 0x7f4aac004d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8002b10 tx=0x7f4ab80331b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f00cb9d9e6d7999 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 1 -- 10.20.193.6:0/1436965060 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f803c070 con 0x7f770815f650 2026-02-22T23:23:27.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: dump: 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 1 -- 10.20.193.6:0/1436965060 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f802fb30 con 0x7f770815f650 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 1 -- 10.20.193.6:0/1436965060 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8037440 con 0x7f770815f650 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770c86f640 10 monclient: dump: 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770fafc640 5 monclient: authenticate success, global_id 4511 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770fafc640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.912+0000 7f770fafc640 4 set_mon_vals no callback set 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 10 monclient: shutdown 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 -- 10.20.193.6:0/1436965060 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f770815f650 msgr2=0x7f770815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 --2- 10.20.193.6:0/1436965060 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f770815f650 0x7f770815fa30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f770808e9e0 tx=0x7f76f802ee60 comp rx=0 tx=0).stop 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7ff09e340640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7ff09e340640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7ff09e340640 10 monclient: monmap: 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.914062+0000 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.914062+0000 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 -- 10.20.193.6:0/1436965060 shutdown_connections 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 --2- 10.20.193.6:0/1436965060 >> v2:10.20.193.6:3300/0 conn(0x7f77080ceca0 0x7f77080d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 --2- 10.20.193.6:0/1436965060 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f770815f650 0x7f770815fa30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 -- 10.20.193.6:0/1436965060 >> 10.20.193.6:0/1436965060 conn(0x7f7708072fc0 msgr2=0x7f77080733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 -- 10.20.193.6:0/1436965060 shutdown_connections 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f770fafc640 1 -- 10.20.193.6:0/1436965060 wait complete. 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f7d884a9640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f7d884a9640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7f7d884a9640 10 monclient: monmap: 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.914249+0000 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.914249+0000 2026-02-22T23:23:27.924 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 1 -- 10.20.193.6:0/4130439526 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c046070 con 0x7fd29015fa30 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 10 monclient: dump: 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 1 -- 10.20.193.6:0/4130439526 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c041890 con 0x7fd29015fa30 2026-02-22T23:23:27.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd2857fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7ff09e340640 1 Processor -- start 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.913+0000 7fd286ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 1 -- start start 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient: init 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 5 monclient: authenticate success, global_id 4483 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 monclient: _renew_subs 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd2857fa640 1 -- 10.20.193.6:0/4130439526 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c03c040 con 0x7fd29015fa30 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2900716c0 con 0x7fd29015fa30 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd2857fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd2857fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd2857fa640 10 monclient: dump: 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 client.4483.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 monclient: _renew_subs 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd290071c40 con 0x7fd29015fa30 2026-02-22T23:23:27.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 1 librados: init done 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 10 monclient: monmap: 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f7d884a9640 1 Processor -- start 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 1 -- 10.20.193.6:0/1333614327 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e002fd00 con 0x7f99f015fad0 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: dump: 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 1 -- 10.20.193.6:0/1333614327 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e00373f0 con 0x7f99f015fad0 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 1 -- 10.20.193.6:0/1333614327 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e00377c0 con 0x7f99f015fad0 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99eeffd640 10 monclient: dump: 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 5 monclient: authenticate success, global_id 4477 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 4 set_mon_vals no callback set 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f7d884a9640 1 -- start start 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f7d884a9640 10 monclient: init 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 librados: starting msgr at 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 librados: starting objecter 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 Processor -- start 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 -- start start 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 librados: setting wanted keys 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 librados: calling monclient init 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 10 monclient: init 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fb279d74640 1 -- 10.20.193.6:0/3374040633 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3593397379 0 0) 0x7fb27c0c7e90 con 0x7fb27c0c99a0 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fb279d74640 10 monclient(hunting): _negotiate 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fb279d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:27.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fb279d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fb279d74640 10 monclient(hunting): my global_id is 4489 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fb279d74640 1 -- 10.20.193.6:0/3374040633 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb268003620 con 0x7fb27c0c99a0 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient: _add_conns ranks=[5,0,3,4,1,2] 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff098155e70 0x7ff098074120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient(hunting): picked mon.noname-c con 0x7ff098155e70 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff098074d60 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff098156250 0x7ff09815d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient(hunting): picked mon.noname-a con 0x7ff098156250 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0980ccb50 con 0x7ff098074d60 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff0980cc920 con 0x7ff098156250 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff0980cce30 con 0x7ff098155e70 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7ff09e340640 10 monclient(hunting): authenticate will time out at 1082.641753s 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 10 monclient: shutdown 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 1 -- 10.20.193.6:0/1333614327 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 msgr2=0x7f99f00ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1333614327 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 0x7f99f00ccf70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009e50 tx=0x7f99e002f2a0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd290074ea0 con 0x7fd29015fa30 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 1 -- 10.20.193.6:0/1333614327 shutdown_connections 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1333614327 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 0x7f99f00ccf70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 1 -- 10.20.193.6:0/1333614327 >> 10.20.193.6:0/1333614327 conn(0x7f99f00ce730 msgr2=0x7f99f00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 1 -- 10.20.193.6:0/1333614327 shutdown_connections 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f77080ceca0 0x7f77081f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 10 monclient(hunting): picked mon.noname-b con 0x7f77080ceca0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f7d884a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f7d884a9640 10 monclient: _add_conns ranks=[3,2,0,4,5,1] 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f7d884a9640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f7d801527f0 0x7f7d80073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f77080d1ea0 0x7f7708158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f7d884a9640 10 monclient(hunting): picked mon.noname-a con 0x7f7d801527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f7d8015d2c0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f99f68b0640 1 -- 10.20.193.6:0/1333614327 wait complete. 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 10 monclient(hunting): picked mon.noname-c con 0x7f77080d1ea0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.914+0000 7f770fafc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f770815f650 0x7f77081e9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7d8015d730 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d80073c20 con 0x7f7d8015d730 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d80073f00 con 0x7f7d8015d2c0 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7d80074b10 con 0x7f7d801527f0 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f7d884a9640 10 monclient(hunting): authenticate will time out at 1082.642041s 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 10 monclient(hunting): picked mon.noname-a con 0x7f770815f650 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f77080cf0f0 con 0x7f770815f650 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f77080c95e0 con 0x7f77080ceca0 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77081f4bf0 con 0x7f77080d1ea0 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f770fafc640 10 monclient(hunting): authenticate will time out at 1082.642046s 2026-02-22T23:23:27.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 1 -- 10.20.193.6:0/1868574122 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180052d0 con 0x7f463015d2c0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: dump: 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 1 -- 10.20.193.6:0/1868574122 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4618005470 con 0x7f463015d2c0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 1 -- 10.20.193.6:0/1868574122 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180038f0 con 0x7f463015d2c0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f4626ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f46257fa640 10 monclient: dump: 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 5 monclient: authenticate success, global_id 4486 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 10 monclient: _renew_subs 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46301f6030 con 0x7f463015d2c0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 10 client.4486.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 10 monclient: _renew_subs 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46301f65f0 con 0x7f463015d2c0 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f463712c640 1 librados: init done 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 10 monclient: monmap: 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 1 librados: starting msgr at 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 1 librados: starting objecter 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 1 Processor -- start 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 1 -- start start 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 1 librados: setting wanted keys 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 1 librados: calling monclient init 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.915+0000 7f99f68b0640 10 monclient: init 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f463712c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f463712c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f463712c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:27.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4630152bd0 con 0x7f463015d2c0 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f99f0073530 0x7f99f015e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): picked mon.noname-c con 0x7f99f0073530 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f99f01527f0 0x7f99f01f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): picked mon.noname-a con 0x7f99f01527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 0x7f99f01f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): picked mon.noname-b con 0x7f99f015fad0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00744c0 con 0x7f99f01527f0 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00741e0 con 0x7f99f015fad0 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00747a0 con 0x7f99f0073530 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.916+0000 7f99f68b0640 10 monclient(hunting): authenticate will time out at 1082.643363s 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7f59bac1a640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7f59bac1a640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7f59bac1a640 10 monclient: monmap: 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.921802+0000 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.921802+0000 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.934 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fd7921d3640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fd7921d3640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fd7921d3640 10 monclient: monmap: 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.921893+0000 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.921893+0000 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7f59bac1a640 1 Processor -- start 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fad2fc58640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fad2fc58640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fad2fc58640 10 monclient: monmap: 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.922250+0000 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.922250+0000 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.935 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fd7921d3640 1 Processor -- start 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7f59bac1a640 1 -- start start 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7f59bac1a640 10 monclient: init 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fd7921d3640 1 -- start start 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.921+0000 7fd7921d3640 10 monclient: init 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fad2fc58640 1 Processor -- start 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fad2fc58640 1 -- start start 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fad2fc58640 10 monclient: init 2026-02-22T23:23:27.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient: _add_conns ranks=[4,2,0,5,1,3] 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fd78c06ff20 0x7fd78c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient(hunting): picked mon.noname-b con 0x7fd78c06ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd78c0c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd78c15a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78c0c7e90 con 0x7fd78c15a5e0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78c0c8170 con 0x7fd78c0c99a0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd78c159a40 con 0x7fd78c06ff20 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fd7921d3640 10 monclient(hunting): authenticate will time out at 1082.649466s 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient: _add_conns ranks=[4,3,1,5,0,2] 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f59b41527f0 0x7f59b4073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient(hunting): picked mon.noname-b con 0x7f59b41527f0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f59b415d2c0 0x7f59b415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient(hunting): picked mon.noname-a con 0x7f59b415d2c0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f59b415d730 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b4073c20 con 0x7f59b415d730 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f59b4074b10 con 0x7f59b415d2c0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f59b4073f00 con 0x7f59b41527f0 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7f59bac1a640 10 monclient(hunting): authenticate will time out at 1082.649496s 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fad2fc58640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fad2fc58640 10 monclient: _add_conns ranks=[1,5,2,0,4,3] 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fad2fc58640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fad281527f0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.922+0000 7fad2fc58640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fad28073530 0x7fad2815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 10 monclient(hunting): picked mon.noname-c con 0x7fad28073530 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fad2815f900 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad280739a0 con 0x7fad281527f0 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad2815fd90 con 0x7fad2815f900 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fad28074b10 con 0x7fad28073530 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.923+0000 7fad2fc58640 10 monclient(hunting): authenticate will time out at 1082.650204s 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.937+0000 7fda4b555640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.937+0000 7fda4b555640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.937+0000 7fda4b555640 10 monclient: monmap: 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.938353+0000 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.938353+0000 2026-02-22T23:23:27.938 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 1 Processor -- start 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 1 -- start start 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 10 monclient: init 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fac8bda9640 10 monclient: get_monmap_and_config 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fac8bda9640 10 monclient: build_initial_monmap 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fac8bda9640 10 monclient: monmap: 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:27.939 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:27.939389+0000 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:27.939389+0000 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 10 monclient: _add_conns ranks=[4,5,2,3,0,1] 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fda441527f0 0x7fda44073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 10 monclient(hunting): picked mon.noname-b con 0x7fda441527f0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fda4415d2c0 0x7fda4415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.938+0000 7fda4b555640 10 monclient(hunting): picked mon.noname-c con 0x7fda4415d2c0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fda4415d730 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda44073c20 con 0x7fda4415d730 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fda44074b10 con 0x7fda441527f0 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fda44073f00 con 0x7fda4415d2c0 2026-02-22T23:23:27.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fda4b555640 10 monclient(hunting): authenticate will time out at 1082.666122s 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 1 Processor -- start 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 1 -- start start 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient: init 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient: _add_conns ranks=[5,2,1,4,0,3] 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fac841527f0 0x7fac84073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient(hunting): picked mon.noname-c con 0x7fac841527f0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fac8415d2c0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fac8415d730 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac84073c20 con 0x7fac8415d730 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac84073f00 con 0x7fac8415d2c0 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fac84074b10 con 0x7fac841527f0 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:27.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:27.939+0000 7fac8bda9640 10 monclient(hunting): authenticate will time out at 1082.666984s 2026-02-22T23:23:28.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.000+0000 7f5877fff640 1 -- 10.20.193.6:0/2784236733 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5890053060 con 0x7f589c14bdb0 2026-02-22T23:23:28.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.000+0000 7f5877fff640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f586c078e50 0x7f586c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.000+0000 7f589b7fe640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f586c078e50 0x7f586c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.000+0000 7f589b7fe640 10 monclient: get_auth_request con 0x7f586c078e50 auth_method 0 2026-02-22T23:23:28.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.001+0000 7f589b7fe640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f586c078e50 0x7f586c07b310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5888011310 tx=0x7f5888009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.001+0000 7f5877fff640 10 client.4449.objecter ms_handle_connect 0x7f586c078e50 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c007de0 con 0x7fad10004900 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 10 monclient: dump: 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.009 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad0c005bf0 con 0x7fad10004900 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 1 -- 10.20.193.6:0/1347071558 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c007070 con 0x7fad10004900 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 4 set_mon_vals no callback set 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad21573640 10 monclient: dump: 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.010 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 10 monclient: shutdown 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 1 -- 10.20.193.6:0/1347071558 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad10004900 msgr2=0x7fad10004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad10004900 0x7fad10004d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00c270 tx=0x7fad0c02f5b0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 1 -- 10.20.193.6:0/1347071558 shutdown_connections 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 1 --2- 10.20.193.6:0/1347071558 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad10004900 0x7fad10004d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 1 -- 10.20.193.6:0/1347071558 >> 10.20.193.6:0/1347071558 conn(0x7fad240ce730 msgr2=0x7fad240ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 1 -- 10.20.193.6:0/1347071558 shutdown_connections 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.008+0000 7fad28c61640 1 -- 10.20.193.6:0/1347071558 wait complete. 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 10 monclient: monmap: 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 1 librados: starting msgr at 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 1 librados: starting objecter 2026-02-22T23:23:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 1 Processor -- start 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 1 -- start start 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 1 librados: setting wanted keys 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 1 librados: calling monclient init 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.009+0000 7fad28c61640 10 monclient: init 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 1 -- 10.20.193.6:0/2784236733 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f58900027c0 con 0x7f589c14bdb0 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 10 client.4449.objecter ms_dispatch2 0x7f589c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 3 client.4449.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 3 client.4449.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 20 client.4449.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 1 -- 10.20.193.6:0/2784236733 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f589c14c240 con 0x7f589c14bdb0 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7f5877fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10004900 0x7fad24150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): picked mon.noname-a con 0x7fad10004900 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad24150f80 0x7fad240c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): picked mon.noname-c con 0x7fad24150f80 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad24159c40 0x7fad240c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): picked mon.noname-b con 0x7fad24159c40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad240744c0 con 0x7fad10004900 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad240741e0 con 0x7fad24159c40 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad240747a0 con 0x7fad24150f80 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.010+0000 7fad28c61640 10 monclient(hunting): authenticate will time out at 1082.737406s 2026-02-22T23:23:28.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.016+0000 7f770e072640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f770815f650 0x7f77081e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.016+0000 7f5aa1ffb640 1 -- 10.20.193.6:0/3039883624 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5ab001b3b0 con 0x7f5ab415d730 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f5aa1ffb640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5a84078e50 0x7f5a8407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f770815f650 0x7f77081e9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58966/0 (socket says 10.20.193.6:58966) 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 1 -- 10.20.193.6:0/738428046 learned_addr learned my addr 10.20.193.6:0/738428046 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 10 monclient(hunting): get_auth_request con 0x7f770815f650 auth_method 0 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f5ab9160640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5a84078e50 0x7f5a8407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f5ab9160640 10 monclient: get_auth_request con 0x7f5a84078e50 auth_method 0 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7ff09c8b6640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff098156250 0x7ff09815d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f5ab9160640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5a84078e50 0x7f5a8407b310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006b90 tx=0x7f5aa4006b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f5aa1ffb640 10 client.4514.objecter ms_handle_connect 0x7f5a84078e50 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.017+0000 7f770e072640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7ff09c8b6640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff098156250 0x7ff09815d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58958/0 (socket says 10.20.193.6:58958) 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7ff09c8b6640 1 -- 10.20.193.6:0/3039507265 learned_addr learned my addr 10.20.193.6:0/3039507265 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7ff09c8b6640 10 monclient(hunting): get_auth_request con 0x7ff098156250 auth_method 0 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7ff09c8b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7ff09c8b6640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7ff09c8b6640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7f5aa1ffb640 1 -- 10.20.193.6:0/3039883624 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f5ab005dcc0 con 0x7f5ab415d730 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7f5aa1ffb640 10 client.4514.objecter ms_dispatch2 0x7f5ab4000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7f5aa1ffb640 3 client.4514.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7f5aa1ffb640 3 client.4514.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.018+0000 7f5aa1ffb640 20 client.4514.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 10 monclient(hunting): handle_auth_done global_id 4461 payload 824 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 1 -- 10.20.193.6:0/738428046 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f77080d1ea0 msgr2=0x7f7708158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f77080d1ea0 0x7f7708158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 1 -- 10.20.193.6:0/738428046 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f77080ceca0 msgr2=0x7f77081f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f77080ceca0 0x7f77081f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.019+0000 7f770e072640 1 -- 10.20.193.6:0/738428046 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77081574a0 con 0x7f770815f650 2026-02-22T23:23:28.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.020+0000 7ff09c8b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.020+0000 7ff09c8b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.020+0000 7ff09c8b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.020+0000 7fd2857fa640 1 -- 10.20.193.6:0/4130439526 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd28c052060 con 0x7fd29015fa30 2026-02-22T23:23:28.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.021+0000 7fd2857fa640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd260078e50 0x7fd26007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.021+0000 7fd294e78640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd260078e50 0x7fd26007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.021+0000 7fd294e78640 10 monclient: get_auth_request con 0x7fd260078e50 auth_method 0 2026-02-22T23:23:28.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.021+0000 7fb27ad76640 1 --2- 10.20.193.6:0/3374040633 >> v2:10.20.193.6:3300/0 conn(0x7fb27c06ff20 0x7fb27c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.021+0000 7f770e072640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f770815f650 0x7f77081e9340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f770400cb70 tx=0x7f770400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.021+0000 7f76f67fc640 10 client.?.objecter ms_handle_connect 0x7f770815f650 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.021+0000 7f76f67fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7fd294e78640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd260078e50 0x7fd26007b310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd280009750 tx=0x7fd280006ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7fd2857fa640 10 client.4483.objecter ms_handle_connect 0x7fd260078e50 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 10 monclient(hunting): handle_auth_done global_id 4464 payload 274 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 1 -- 10.20.193.6:0/3039507265 >> v2:10.20.193.80:3300/0 conn(0x7ff098155e70 msgr2=0x7ff098074120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 1 --2- 10.20.193.6:0/3039507265 >> v2:10.20.193.80:3300/0 conn(0x7ff098155e70 0x7ff098074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 1 -- 10.20.193.6:0/3039507265 >> v1:10.20.193.6:6789/0 conn(0x7ff098074d60 legacy=0x7ff0980cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7ff09c8b6640 1 -- 10.20.193.6:0/3039507265 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0980cd110 con 0x7ff098156250 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7fb27ad76640 10 monclient(hunting): get_auth_request con 0x7fb27c06ff20 auth_method 0 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7fb27ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7fb27ad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.022+0000 7fb27ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.023+0000 7fb27ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.023+0000 7fb27ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.023+0000 7fb27ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.023+0000 7f7d8621e640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f7d801527f0 0x7f7d80073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7fd2857fa640 1 -- 10.20.193.6:0/4130439526 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fd28c0027c0 con 0x7fd29015fa30 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7fd2857fa640 10 client.4483.objecter ms_dispatch2 0x7fd290000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7fd2857fa640 3 client.4483.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7fd2857fa640 3 client.4483.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7fd2857fa640 20 client.4483.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7f5aa1ffb640 1 -- 10.20.193.6:0/3039883624 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5ab0021020 con 0x7f5ab415d730 2026-02-22T23:23:28.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7f5aa1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.024+0000 7f5aa1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.025+0000 7ff09c8b6640 1 --2- 10.20.193.6:0/3039507265 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 0x7ff09815d460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0980c5300 tx=0x7ff08c02f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0d3f670348c672 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 10 monclient(hunting): handle_auth_done global_id 4467 payload 274 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 1 -- 10.20.193.6:0/3374040633 >> v1:10.20.193.80:6789/0 conn(0x7fb27c15a5e0 legacy=0x7fb27c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 1 -- 10.20.193.6:0/3374040633 >> v1:10.20.193.40:6789/0 conn(0x7fb27c0c99a0 legacy=0x7fb27c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7fb27ad76640 1 -- 10.20.193.6:0/3374040633 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c0c8450 con 0x7fb27c06ff20 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7f7d8621e640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f7d801527f0 0x7f7d80073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58970/0 (socket says 10.20.193.6:58970) 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7f7d8621e640 1 -- 10.20.193.6:0/1646065137 learned_addr learned my addr 10.20.193.6:0/1646065137 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7f7d8621e640 10 monclient(hunting): get_auth_request con 0x7f7d801527f0 auth_method 0 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7f7d8621e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7f7d8621e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.026+0000 7f7d8621e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7fda48ac9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fda4415d2c0 0x7fda4415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7f7d8621e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7f7d8621e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7f7d8621e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7fda48ac9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fda4415d2c0 0x7fda4415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43592/0 (socket says 10.20.193.6:43592) 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7fda48ac9640 1 -- 10.20.193.6:0/3304692401 learned_addr learned my addr 10.20.193.6:0/3304692401 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7fda48ac9640 10 monclient(hunting): get_auth_request con 0x7fda4415d2c0 auth_method 0 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7fda48ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7fda48ac9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.027+0000 7fda48ac9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f59abfff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f59b415d2c0 0x7f59b415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8002b50 con 0x7f4aac004900 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 10 monclient: dump: 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.029 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab8037070 con 0x7f4aac004900 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 1 -- 10.20.193.6:0/3178618509 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8033e50 con 0x7f4aac004900 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4ac8b57640 10 monclient: dump: 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4acbde4640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7f4acbde4640 4 set_mon_vals no callback set 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7fda48ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7fda48ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7fda48ac9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.028+0000 7fb27ad76640 1 --2- 10.20.193.6:0/3374040633 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 0x7fb27c070300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb2700098d0 tx=0x7fb27002ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295f4dc9ba5b467b server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f99effff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f99f0073530 0x7f99f015e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 10 monclient: shutdown 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 1 -- 10.20.193.6:0/3178618509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4aac004900 msgr2=0x7f4aac004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4aac004900 0x7f4aac004d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8002b10 tx=0x7f4ab80331b0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 1 -- 10.20.193.6:0/3178618509 shutdown_connections 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3178618509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4aac004900 0x7f4aac004d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 1 -- 10.20.193.6:0/3178618509 >> 10.20.193.6:0/3178618509 conn(0x7f4ac40cc020 msgr2=0x7f4ac40cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 1 -- 10.20.193.6:0/3178618509 shutdown_connections 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 1 -- 10.20.193.6:0/3178618509 wait complete. 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f4acbde4640 10 monclient: monmap: 2026-02-22T23:23:28.031 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 10 monclient(hunting): handle_auth_done global_id 4520 payload 274 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 1 -- 10.20.193.6:0/3304692401 >> v2:10.20.193.40:3300/0 conn(0x7fda441527f0 msgr2=0x7fda44073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 1 --2- 10.20.193.6:0/3304692401 >> v2:10.20.193.40:3300/0 conn(0x7fda441527f0 0x7fda44073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 1 -- 10.20.193.6:0/3304692401 >> v1:10.20.193.80:6789/0 conn(0x7fda4415d730 legacy=0x7fda440ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 10 monclient(hunting): handle_auth_done global_id 4470 payload 274 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7fda48ac9640 1 -- 10.20.193.6:0/3304692401 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda440741e0 con 0x7fda4415d2c0 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 1 -- 10.20.193.6:0/1646065137 >> v1:10.20.193.80:6789/0 conn(0x7f7d8015d2c0 legacy=0x7f7d8015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 1 -- 10.20.193.6:0/1646065137 >> v1:10.20.193.6:6789/0 conn(0x7f7d8015d730 legacy=0x7f7d800ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.029+0000 7f7d8621e640 1 -- 10.20.193.6:0/1646065137 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d800741e0 con 0x7f7d801527f0 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 librados: starting msgr at 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 librados: starting objecter 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 Processor -- start 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 -- start start 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 librados: setting wanted keys 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 librados: calling monclient init 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient: init 2026-02-22T23:23:28.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f59abfff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f59b415d2c0 0x7f59b415f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:58988/0 (socket says 10.20.193.6:58988) 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f59abfff640 1 -- 10.20.193.6:0/3935955809 learned_addr learned my addr 10.20.193.6:0/3935955809 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f59abfff640 10 monclient(hunting): get_auth_request con 0x7f59b415d2c0 auth_method 0 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f59abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f59abfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f59abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f99effff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f99f0073530 0x7f99f015e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43572/0 (socket says 10.20.193.6:43572) 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f99effff640 1 -- 10.20.193.6:0/1558078321 learned_addr learned my addr 10.20.193.6:0/1558078321 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f99effff640 10 monclient(hunting): get_auth_request con 0x7f99f0073530 auth_method 0 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f99effff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f99effff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f99effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac004900 0x7f4ac4151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): picked mon.noname-a con 0x7f4aac004900 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ac4151aa0 0x7f4ac40c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): picked mon.noname-c con 0x7f4ac4151aa0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ac41f2180 0x7f4ac40c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): picked mon.noname-b con 0x7f4ac41f2180 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40c6d20 con 0x7f4aac004900 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40c9020 con 0x7f4ac41f2180 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40748e0 con 0x7f4ac4151aa0 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.030+0000 7f4acbde4640 10 monclient(hunting): authenticate will time out at 1082.757571s 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7f59abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7f59abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7f59abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7f99ef7fe640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f99f01527f0 0x7f99f01f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7fd2857fa640 1 -- 10.20.193.6:0/4130439526 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd290151270 con 0x7fd29015fa30 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7fd2857fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7f99effff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7f99effff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7fd2857fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.031+0000 7f99effff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 10 monclient(hunting): handle_auth_done global_id 4523 payload 824 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 1 -- 10.20.193.6:0/1558078321 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 msgr2=0x7f99f01f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 0x7f99f01f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 1 -- 10.20.193.6:0/1558078321 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f99f01527f0 msgr2=0x7f99f01f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f99f01527f0 0x7f99f01f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f99effff640 1 -- 10.20.193.6:0/1558078321 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f00c7450 con 0x7f99f0073530 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 10 monclient(hunting): handle_auth_done global_id 4473 payload 274 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 1 -- 10.20.193.6:0/3935955809 >> v2:10.20.193.40:3300/0 conn(0x7f59b41527f0 msgr2=0x7f59b4073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 1 --2- 10.20.193.6:0/3935955809 >> v2:10.20.193.40:3300/0 conn(0x7f59b41527f0 0x7f59b4073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 1 -- 10.20.193.6:0/3935955809 >> v1:10.20.193.40:6789/0 conn(0x7f59b415d730 legacy=0x7f59b40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f59abfff640 1 -- 10.20.193.6:0/3935955809 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b40741e0 con 0x7f59b415d2c0 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7f7d8621e640 1 --2- 10.20.193.6:0/1646065137 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d801527f0 0x7f7d80073530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009970 tx=0x7f7d7002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d16f013c57c11af server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.033+0000 7fda48ac9640 1 --2- 10.20.193.6:0/3304692401 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda4415d2c0 0x7fda4415f3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda4408f510 tx=0x7fda3802eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=257e5ad4ff01ea54 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.034+0000 7fad2d1cc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fad28073530 0x7fad2815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7f59abfff640 1 --2- 10.20.193.6:0/3935955809 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b415d2c0 0x7f59b415f3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59b406e9a0 tx=0x7f599c02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd685255624d00d0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fd78bfff640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fd78c15a5e0 0x7fd78c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58876/0 (socket says 10.20.193.6:58876) 2026-02-22T23:23:28.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fd78bfff640 1 -- 10.20.193.6:0/605487740 learned_addr learned my addr 10.20.193.6:0/605487740 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fad2d1cc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fad28073530 0x7fad2815f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43584/0 (socket says 10.20.193.6:43584) 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1771828231 learned_addr learned my addr 10.20.193.6:0/1771828231 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fad2d1cc640 10 monclient(hunting): get_auth_request con 0x7fad28073530 auth_method 0 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fad2d1cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fad2d1cc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.035+0000 7fad2d1cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fac89b1e640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fac841527f0 0x7fac84073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7f99effff640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f99f0073530 0x7f99f015e4d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99e400b570 tx=0x7f99e400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7f99ecff9640 10 client.?.objecter ms_handle_connect 0x7f99f0073530 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7f99ecff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.038 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fac89b1e640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fac841527f0 0x7fac84073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43594/0 (socket says 10.20.193.6:43594) 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fac89b1e640 1 -- 10.20.193.6:0/2947240160 learned_addr learned my addr 10.20.193.6:0/2947240160 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fac89b1e640 10 monclient(hunting): get_auth_request con 0x7fac841527f0 auth_method 0 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fac89b1e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fac89b1e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.036+0000 7fac89b1e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad22575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10004900 0x7fad24150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_done global_id 4526 payload 274 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1771828231 >> v1:10.20.193.80:6789/0 conn(0x7fad2815f900 legacy=0x7fad280ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1771828231 >> v1:10.20.193.40:6789/0 conn(0x7fad281527f0 legacy=0x7fad2815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.037+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1771828231 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2815d9d0 con 0x7fad28073530 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fac89b1e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fac89b1e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fac89b1e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fad21d74640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad24150f80 0x7fad240c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fad22575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10004900 0x7fad24150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39736/0 (socket says 10.20.193.6:39736) 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fad22575640 1 -- 10.20.193.6:0/2911049934 learned_addr learned my addr 10.20.193.6:0/2911049934 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fad22575640 10 monclient(hunting): get_auth_request con 0x7fad10004900 auth_method 0 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fad22575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fad22575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.038+0000 7fad22575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 10 monclient(hunting): handle_auth_done global_id 4529 payload 274 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 1 -- 10.20.193.6:0/2947240160 >> v1:10.20.193.80:6789/0 conn(0x7fac8415d2c0 legacy=0x7fac8415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 1 -- 10.20.193.6:0/2947240160 >> v1:10.20.193.40:6789/0 conn(0x7fac8415d730 legacy=0x7fac840ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fac89b1e640 1 -- 10.20.193.6:0/2947240160 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac840741e0 con 0x7fac841527f0 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fad21d74640 10 monclient(hunting): get_auth_request con 0x7fad24150f80 auth_method 0 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fad21d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fad21d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.039+0000 7fad21d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.040+0000 7fad22575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.040+0000 7fad22575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.040+0000 7fad22575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.040+0000 7fad21d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.040+0000 7fad21d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.040+0000 7fad21d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.041+0000 7fad2d1cc640 1 --2- 10.20.193.6:0/1771828231 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad28073530 0x7fad2815f3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009970 tx=0x7fad1c02eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60fe3c60a2648121 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.041+0000 7fac89b1e640 1 --2- 10.20.193.6:0/2947240160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 0x7fac84073530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac74009970 tx=0x7fac7402eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2251650792e5bd server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 10 monclient(hunting): handle_auth_done global_id 4479 payload 824 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 1 -- 10.20.193.6:0/2911049934 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad24150f80 msgr2=0x7fad240c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad24150f80 0x7fad240c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 1 -- 10.20.193.6:0/2911049934 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad24159c40 msgr2=0x7fad240c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad24159c40 0x7fad240c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fad22575640 1 -- 10.20.193.6:0/2911049934 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2415a200 con 0x7fad10004900 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 1 -- 10.20.193.6:0/3374040633 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27003c070 con 0x7fb27c06ff20 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: dump: 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 1 -- 10.20.193.6:0/3374040633 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27002fb00 con 0x7fb27c06ff20 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb28102b640 5 monclient: authenticate success, global_id 4467 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb28102b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 1 -- 10.20.193.6:0/3374040633 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270037440 con 0x7fb27c06ff20 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb28102b640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb28102b640 4 set_mon_vals no callback set 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.042+0000 7fb279d74640 10 monclient: dump: 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.044 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 10 monclient: shutdown 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 1 -- 10.20.193.6:0/3374040633 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 msgr2=0x7fb27c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 1 --2- 10.20.193.6:0/3374040633 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 0x7fb27c070300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb2700098d0 tx=0x7fb27002ee80 comp rx=0 tx=0).stop 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 1 -- 10.20.193.6:0/3374040633 shutdown_connections 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 1 --2- 10.20.193.6:0/3374040633 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 0x7fb27c070300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 1 -- 10.20.193.6:0/3374040633 >> 10.20.193.6:0/3374040633 conn(0x7fb27c0cd780 msgr2=0x7fb27c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 1 -- 10.20.193.6:0/3374040633 shutdown_connections 2026-02-22T23:23:28.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7fb28102b640 1 -- 10.20.193.6:0/3374040633 wait complete. 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 1 -- 10.20.193.6:0/3039507265 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c02fcd0 con 0x7ff098156250 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: dump: 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 1 -- 10.20.193.6:0/3039507265 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c0373f0 con 0x7ff098156250 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 1 -- 10.20.193.6:0/3039507265 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0377c0 con 0x7ff098156250 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff096ffd640 10 monclient: dump: 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.046 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff09e340640 5 monclient: authenticate success, global_id 4464 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff09e340640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.043+0000 7ff09e340640 4 set_mon_vals no callback set 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 10 monclient: monmap: 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 10 monclient: shutdown 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 -- 10.20.193.6:0/3039507265 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 msgr2=0x7ff09815d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 --2- 10.20.193.6:0/3039507265 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 0x7ff09815d460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff0980c5300 tx=0x7ff08c02f270 comp rx=0 tx=0).stop 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 -- 10.20.193.6:0/3039507265 shutdown_connections 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 --2- 10.20.193.6:0/3039507265 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 0x7ff09815d460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 --2- 10.20.193.6:0/3039507265 >> v2:10.20.193.80:3300/0 conn(0x7ff098155e70 0x7ff098074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 -- 10.20.193.6:0/3039507265 >> 10.20.193.6:0/3039507265 conn(0x7ff0980ce700 msgr2=0x7ff0980ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fd78a7fc640 1 -- 10.20.193.6:0/605487740 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160609116 0 0) 0x7fd78c0c7e90 con 0x7fd78c15a5e0 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fd78a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fd78a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fd78a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fd78a7fc640 10 monclient(hunting): my global_id is 4476 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fd78a7fc640 1 -- 10.20.193.6:0/605487740 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd770003620 con 0x7fd78c15a5e0 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 1 librados: starting msgr at 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 1 librados: starting objecter 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 -- 10.20.193.6:0/3039507265 shutdown_connections 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7ff09e340640 1 -- 10.20.193.6:0/3039507265 wait complete. 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 1 Processor -- start 2026-02-22T23:23:28.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 1 -- start start 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 1 librados: setting wanted keys 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 1 librados: calling monclient init 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7fb28102b640 10 monclient: init 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.044+0000 7f4ac9b59640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac004900 0x7f4ac4151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient: monmap: 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.049 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 0x7fb27c0c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): picked mon.noname-a con 0x7fb27c06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb27c0c99a0 0x7fb27c1f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): picked mon.noname-b con 0x7fb27c0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb27c15a5e0 0x7fb27c0bea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): picked mon.noname-c con 0x7fb27c15a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c150520 con 0x7fb27c06ff20 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c0c8a10 con 0x7fb27c0c99a0 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1506a0 con 0x7fb27c15a5e0 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fb28102b640 10 monclient(hunting): authenticate will time out at 1082.772191s 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 librados: starting msgr at 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 librados: starting objecter 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 Processor -- start 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 -- start start 2026-02-22T23:23:28.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 librados: setting wanted keys 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 librados: calling monclient init 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient: init 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fad22575640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10004900 0x7fad24150a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b5a0 tx=0x7fad1800ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fad0affd640 10 client.?.objecter ms_handle_connect 0x7fad10004900 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff098074d60 0x7ff098142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): picked mon.noname-c con 0x7ff098074d60 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7fad0affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff098155e70 0x7ff098144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): picked mon.noname-b con 0x7ff098155e70 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 0x7ff09814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): picked mon.noname-a con 0x7ff098156250 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff0980cd9b0 con 0x7ff098156250 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff0980cd6d0 con 0x7ff098155e70 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff098074660 con 0x7ff098074d60 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.045+0000 7ff09e340640 10 monclient(hunting): authenticate will time out at 1082.772979s 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.046+0000 7f4ac9b59640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac004900 0x7f4ac4151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39740/0 (socket says 10.20.193.6:39740) 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.046+0000 7f4ac9b59640 1 -- 10.20.193.6:0/3351743640 learned_addr learned my addr 10.20.193.6:0/3351743640 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.046+0000 7f4ac9b59640 10 monclient(hunting): get_auth_request con 0x7f4aac004900 auth_method 0 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.046+0000 7f4ac9b59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.046+0000 7f4ac9b59640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.046+0000 7f4ac9b59640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f4ac9b59640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f4ac9b59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f4ac9b59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 1 -- 10.20.193.6:0/3935955809 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c03c070 con 0x7f59b415d2c0 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: dump: 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 1 -- 10.20.193.6:0/3935955809 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c02fb40 con 0x7f59b415d2c0 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 1 -- 10.20.193.6:0/3935955809 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c037440 con 0x7f59b415d2c0 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59ab7fe640 10 monclient: dump: 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.053 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59bac1a640 5 monclient: authenticate success, global_id 4473 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59bac1a640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7f59bac1a640 4 set_mon_vals no callback set 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 1 -- 10.20.193.6:0/3304692401 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3803c070 con 0x7fda4415d2c0 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: dump: 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 1 -- 10.20.193.6:0/3304692401 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3802fb40 con 0x7fda4415d2c0 2026-02-22T23:23:28.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 1 -- 10.20.193.6:0/3304692401 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38037440 con 0x7fda4415d2c0 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda33fff640 10 monclient: dump: 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda4b555640 5 monclient: authenticate success, global_id 4520 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda4b555640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.047+0000 7fda4b555640 4 set_mon_vals no callback set 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 10 monclient: shutdown 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 -- 10.20.193.6:0/3935955809 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b415d2c0 msgr2=0x7f59b415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 --2- 10.20.193.6:0/3935955809 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b415d2c0 0x7f59b415f3c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59b406e9a0 tx=0x7f599c02eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 -- 10.20.193.6:0/3935955809 shutdown_connections 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 --2- 10.20.193.6:0/3935955809 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b415d2c0 0x7f59b415f3c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 --2- 10.20.193.6:0/3935955809 >> v2:10.20.193.40:3300/0 conn(0x7f59b41527f0 0x7f59b4073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 -- 10.20.193.6:0/3935955809 >> 10.20.193.6:0/3935955809 conn(0x7f59b40ce730 msgr2=0x7f59b40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 -- 10.20.193.6:0/3935955809 shutdown_connections 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 -- 10.20.193.6:0/3935955809 wait complete. 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 10 monclient: shutdown 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 -- 10.20.193.6:0/3304692401 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda4415d2c0 msgr2=0x7fda4415f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 --2- 10.20.193.6:0/3304692401 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda4415d2c0 0x7fda4415f3c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda4408f510 tx=0x7fda3802eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 -- 10.20.193.6:0/3304692401 shutdown_connections 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 --2- 10.20.193.6:0/3304692401 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda4415d2c0 0x7fda4415f3c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 --2- 10.20.193.6:0/3304692401 >> v2:10.20.193.40:3300/0 conn(0x7fda441527f0 0x7fda44073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 -- 10.20.193.6:0/3304692401 >> 10.20.193.6:0/3304692401 conn(0x7fda440ce730 msgr2=0x7fda440ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 -- 10.20.193.6:0/3304692401 shutdown_connections 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7fda4b555640 1 -- 10.20.193.6:0/3304692401 wait complete. 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 10 monclient: monmap: 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.056 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 librados: starting msgr at 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.048+0000 7f59bac1a640 1 librados: starting objecter 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 Processor -- start 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 10 monclient: monmap: 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 -- start start 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 librados: setting wanted keys 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 librados: calling monclient init 2026-02-22T23:23:28.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient: init 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 librados: starting msgr at 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 librados: starting objecter 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 Processor -- start 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 -- start start 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 librados: setting wanted keys 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 librados: calling monclient init 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 10 monclient: init 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b41527f0 0x7f59b414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): picked mon.noname-a con 0x7f59b41527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59b415d2c0 0x7f59b41f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): picked mon.noname-b con 0x7f59b415d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59b415d730 0x7f59b41f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): picked mon.noname-c con 0x7f59b415d730 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59b4074fa0 con 0x7f59b41527f0 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59b40747a0 con 0x7f59b415d2c0 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f59b4075120 con 0x7f59b415d730 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f59bac1a640 10 monclient(hunting): authenticate will time out at 1082.776583s 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 10 monclient(hunting): handle_auth_done global_id 4482 payload 824 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 1 -- 10.20.193.6:0/3351743640 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ac4151aa0 msgr2=0x7f4ac40c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ac4151aa0 0x7f4ac40c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 1 -- 10.20.193.6:0/3351743640 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ac41f2180 msgr2=0x7f4ac40c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ac41f2180 0x7f4ac40c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7f4ac9b59640 1 -- 10.20.193.6:0/3351743640 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac414fed0 con 0x7f4aac004900 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda441527f0 0x7fda440c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 10 monclient(hunting): picked mon.noname-c con 0x7fda441527f0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fda4415d2c0 0x7fda441f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.049+0000 7fda4b555640 10 monclient(hunting): picked mon.noname-b con 0x7fda4415d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fda4415d730 0x7fda441f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 10 monclient(hunting): picked mon.noname-a con 0x7fda4415d730 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fda44074fa0 con 0x7fda4415d730 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fda440747a0 con 0x7fda4415d2c0 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fda44075120 con 0x7fda441527f0 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fda4b555640 10 monclient(hunting): authenticate will time out at 1082.777090s 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.050+0000 7fac88b1c640 1 -- 10.20.193.6:0/2947240160 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7403c070 con 0x7fac841527f0 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: dump: 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 1 -- 10.20.193.6:0/2947240160 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7402fb60 con 0x7fac841527f0 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 1 -- 10.20.193.6:0/2947240160 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74037440 con 0x7fac841527f0 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac88b1c640 10 monclient: dump: 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 5 monclient: authenticate success, global_id 4529 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 4 set_mon_vals no callback set 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 10 monclient: shutdown 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 1 -- 10.20.193.6:0/2947240160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 msgr2=0x7fac84073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 1 --2- 10.20.193.6:0/2947240160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 0x7fac84073530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fac74009970 tx=0x7fac7402eef0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 1 -- 10.20.193.6:0/2947240160 shutdown_connections 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 1 --2- 10.20.193.6:0/2947240160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 0x7fac84073530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 1 -- 10.20.193.6:0/2947240160 >> 10.20.193.6:0/2947240160 conn(0x7fac840ce730 msgr2=0x7fac840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 1 -- 10.20.193.6:0/2947240160 shutdown_connections 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.051+0000 7fac8bda9640 1 -- 10.20.193.6:0/2947240160 wait complete. 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient: monmap: 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 librados: starting msgr at 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 librados: starting objecter 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 Processor -- start 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 -- start start 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 librados: setting wanted keys 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 librados: calling monclient init 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient: init 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 1 -- 10.20.193.6:0/1558078321 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e4004480 con 0x7f99f0073530 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: dump: 2026-02-22T23:23:28.062 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 1 -- 10.20.193.6:0/1558078321 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e4015070 con 0x7f99f0073530 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ee7fc640 4 set_mon_vals no callback set 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 0x7fac841f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): picked mon.noname-c con 0x7fac841527f0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 1 -- 10.20.193.6:0/1558078321 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e40029c0 con 0x7f99f0073530 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99ecff9640 10 monclient: dump: 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.063 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 5 monclient: authenticate success, global_id 4523 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 10 monclient: _renew_subs 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f00c7730 con 0x7f99f0073530 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fac8415d2c0 0x7fac8414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): picked mon.noname-a con 0x7fac8415d2c0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fac8415d730 0x7fac840c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): picked mon.noname-b con 0x7fac8415d730 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fac84074fa0 con 0x7fac8415d2c0 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fac840747a0 con 0x7fac8415d730 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fac84075120 con 0x7fac841527f0 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7fac8bda9640 10 monclient(hunting): authenticate will time out at 1082.779942s 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 10 client.4523.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 10 monclient: _renew_subs 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f00c7cf0 con 0x7f99f0073530 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.052+0000 7f99f68b0640 1 librados: init done 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7f4ac9b59640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac004900 0x7f4ac4151560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400c770 tx=0x7f4ab400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7f4ab27fc640 10 client.?.objecter ms_handle_connect 0x7f4aac004900 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7f4ab27fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7f99f68b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7f99f68b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7f99f68b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f0074f80 con 0x7f99f0073530 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.053+0000 7fb27b577640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb27c15a5e0 0x7fb27c0bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 1 -- 10.20.193.6:0/738428046 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400ad50 con 0x7f770815f650 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: dump: 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 1 -- 10.20.193.6:0/738428046 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7704003a70 con 0x7f770815f650 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 1 -- 10.20.193.6:0/738428046 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704003e60 con 0x7f770815f650 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 5 monclient: authenticate success, global_id 4461 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f67fc640 10 monclient: dump: 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 10 monclient: _renew_subs 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77081e8580 con 0x7f770815f650 2026-02-22T23:23:28.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f76f7fff640 4 set_mon_vals no callback set 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 10 client.4461.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 10 monclient: _renew_subs 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081e8b40 con 0x7f770815f650 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.054+0000 7f770fafc640 1 librados: init done 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 1 -- 10.20.193.6:0/1646065137 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7003c070 con 0x7f7d801527f0 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: dump: 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 1 -- 10.20.193.6:0/1646065137 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7002fb60 con 0x7f7d801527f0 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 1 -- 10.20.193.6:0/1646065137 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70037440 con 0x7f7d801527f0 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d8521c640 10 monclient: dump: 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.067 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 5 monclient: authenticate success, global_id 4470 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 4 set_mon_vals no callback set 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f770fafc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f770fafc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f770fafc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7708074ea0 con 0x7f770815f650 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 10 monclient: shutdown 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 1 -- 10.20.193.6:0/1646065137 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d801527f0 msgr2=0x7f7d80073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1646065137 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d801527f0 0x7f7d80073530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009970 tx=0x7f7d7002eef0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 1 -- 10.20.193.6:0/1646065137 shutdown_connections 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1646065137 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d801527f0 0x7f7d80073530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 1 -- 10.20.193.6:0/1646065137 >> 10.20.193.6:0/1646065137 conn(0x7f7d800ce730 msgr2=0x7f7d800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 1 -- 10.20.193.6:0/1646065137 shutdown_connections 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.055+0000 7f7d884a9640 1 -- 10.20.193.6:0/1646065137 wait complete. 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 10 monclient: monmap: 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 1 librados: starting msgr at 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 1 librados: starting objecter 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 1 Processor -- start 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 1 -- start start 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 1 librados: setting wanted keys 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 1 librados: calling monclient init 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.056+0000 7f7d884a9640 10 monclient: init 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d801527f0 0x7f7d801f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): picked mon.noname-c con 0x7f7d801527f0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d8015d2c0 0x7f7d8014ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): picked mon.noname-b con 0x7f7d8015d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d8015d730 0x7f7d800c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): picked mon.noname-a con 0x7f7d8015d730 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80074fa0 con 0x7f7d8015d730 2026-02-22T23:23:28.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800747a0 con 0x7f7d8015d2c0 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80075120 con 0x7f7d801527f0 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.057+0000 7f7d884a9640 10 monclient(hunting): authenticate will time out at 1082.784129s 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.059+0000 7fb27ad76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 0x7fb27c0c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 1 -- 10.20.193.6:0/2911049934 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18013020 con 0x7fad10004900 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: dump: 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 1 -- 10.20.193.6:0/2911049934 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad18004500 con 0x7fad10004900 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 1 -- 10.20.193.6:0/2911049934 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad180046b0 con 0x7fad10004900 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 5 monclient: authenticate success, global_id 4479 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad20d72640 4 set_mon_vals no callback set 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 10 monclient: _renew_subs 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2415a4e0 con 0x7fad10004900 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad0affd640 10 monclient: dump: 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 10 client.4479.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 10 monclient: _renew_subs 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2415aaa0 con 0x7fad10004900 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 1 librados: init done 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.063+0000 7fad28c61640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.064+0000 7fad28c61640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.064+0000 7fad28c61640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.064+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad24074fa0 con 0x7fad10004900 2026-02-22T23:23:28.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.098+0000 7f58a1dd6640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-22T23:23:28.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.098+0000 7f58a1dd6640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-22T23:23:28.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.098+0000 7f58a1dd6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.098+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f589c0ca630 con 0x7f589c14bdb0 2026-02-22T23:23:28.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.111+0000 7f5abb3eb640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-22T23:23:28.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.111+0000 7f5abb3eb640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-22T23:23:28.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.111+0000 7f5abb3eb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.111+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7f5ab40738c0 con 0x7f5ab415d730 2026-02-22T23:23:28.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.119+0000 7fd297103640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-22T23:23:28.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.119+0000 7fd297103640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-22T23:23:28.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.119+0000 7fd297103640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.119+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd290151ce0 con 0x7fd29015fa30 2026-02-22T23:23:28.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.141+0000 7f46257fa640 1 -- 10.20.193.6:0/1868574122 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4618003a90 con 0x7f463015d2c0 2026-02-22T23:23:28.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.142+0000 7f46257fa640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4600078e00 0x7f460007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.142+0000 7f4634ea1640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4600078e00 0x7f460007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.142+0000 7f4634ea1640 10 monclient: get_auth_request con 0x7f4600078e00 auth_method 0 2026-02-22T23:23:28.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.142+0000 7f46257fa640 10 client.4486.objecter ms_handle_connect 0x7f4600078e00 2026-02-22T23:23:28.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.142+0000 7f4634ea1640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4600078e00 0x7f460007b2c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46200098a0 tx=0x7f4620006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.144+0000 7f46257fa640 1 -- 10.20.193.6:0/1868574122 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f461805e130 con 0x7f463015d2c0 2026-02-22T23:23:28.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.144+0000 7f46257fa640 10 client.4486.objecter ms_dispatch2 0x7f4630000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.144+0000 7f46257fa640 3 client.4486.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.144+0000 7f46257fa640 3 client.4486.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.144+0000 7f46257fa640 20 client.4486.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.150+0000 7f46257fa640 1 -- 10.20.193.6:0/1868574122 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f46180627f0 con 0x7f463015d2c0 2026-02-22T23:23:28.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.150+0000 7f46257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.150+0000 7f46257fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.152+0000 7fb27b577640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb27c15a5e0 0x7fb27c0bea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58536/0 (socket says 10.20.193.6:58536) 2026-02-22T23:23:28.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.152+0000 7fb27b577640 1 -- 10.20.193.6:0/88144848 learned_addr learned my addr 10.20.193.6:0/88144848 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.152+0000 7fb27b577640 10 monclient(hunting): get_auth_request con 0x7fb27c15a5e0 auth_method 0 2026-02-22T23:23:28.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.152+0000 7fb27b577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.152+0000 7fb27b577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.152+0000 7fb27b577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.152+0000 7ff097fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff098074d60 0x7ff098142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7fb27b577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7fb27b577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7fb27b577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7ff097fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff098074d60 0x7ff098142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58548/0 (socket says 10.20.193.6:58548) 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7ff097fff640 1 -- 10.20.193.6:0/2089095073 learned_addr learned my addr 10.20.193.6:0/2089095073 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7ff097fff640 10 monclient(hunting): get_auth_request con 0x7ff098074d60 auth_method 0 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7ff097fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7ff097fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.153+0000 7ff097fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 10 monclient(hunting): handle_auth_done global_id 4538 payload 824 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 1 -- 10.20.193.6:0/88144848 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb27c0c99a0 msgr2=0x7fb27c1f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb27c0c99a0 0x7fb27c1f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 1 -- 10.20.193.6:0/88144848 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 msgr2=0x7fb27c0c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 0x7fb27c0c04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fb27b577640 1 -- 10.20.193.6:0/88144848 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c0c0be0 con 0x7fb27c15a5e0 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7ff097fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7ff097fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7ff097fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fd2857fa640 1 -- 10.20.193.6:0/4130439526 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd28c087ae0 con 0x7fd29015fa30 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fd2857fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.155+0000 7fd2857fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.156 INFO:teuthology.orchestra.run.trial006.stdout:90194313218 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 10 monclient(hunting): handle_auth_done global_id 4541 payload 824 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 1 -- 10.20.193.6:0/2089095073 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff098155e70 msgr2=0x7ff098144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff098155e70 0x7ff098144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 1 -- 10.20.193.6:0/2089095073 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 msgr2=0x7ff09814d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 0x7ff09814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7ff097fff640 1 -- 10.20.193.6:0/2089095073 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff098143680 con 0x7ff098074d60 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 10 librados: watch_flush enter 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 10 librados: watch_flush exit 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 20 client.4483.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 10 client.4483.objecter successfully canceled tick 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd260078e50 msgr2=0x7fd26007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd260078e50 0x7fd26007b310 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd280009750 tx=0x7fd280006ca0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 10 monclient: shutdown 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd29015fa30 msgr2=0x7fd290158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd29015fa30 0x7fd290158b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd28c030fd0 tx=0x7fd28c004b40 comp rx=0 tx=0).stop 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 shutdown_connections 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd260078e50 0x7fd26007b310 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd29015fa30 0x7fd290158b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd29015f650 0x7fd290201d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 --2- 10.20.193.6:0/4130439526 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd2900c9b00 0x7fd290150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 >> 10.20.193.6:0/4130439526 conn(0x7fd290072fc0 msgr2=0x7fd29014eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 shutdown_connections 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 -- 10.20.193.6:0/4130439526 wait complete. 2026-02-22T23:23:28.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fd297103640 1 librados: shutdown 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 1 -- 10.20.193.6:0/1771828231 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c03c070 con 0x7fad28073530 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: dump: 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 1 -- 10.20.193.6:0/1771828231 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1c02fb60 con 0x7fad28073530 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 1 -- 10.20.193.6:0/1771828231 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c037440 con 0x7fad28073530 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2fc58640 5 monclient: authenticate success, global_id 4526 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2fc58640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2fc58640 4 set_mon_vals no callback set 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.157+0000 7fad2c9cb640 10 monclient: dump: 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.160 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 10 monclient: shutdown 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 1 -- 10.20.193.6:0/1771828231 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad28073530 msgr2=0x7fad2815f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1771828231 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad28073530 0x7fad2815f3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009970 tx=0x7fad1c02eef0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 1 -- 10.20.193.6:0/1771828231 shutdown_connections 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1771828231 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad28073530 0x7fad2815f3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 1 -- 10.20.193.6:0/1771828231 >> 10.20.193.6:0/1771828231 conn(0x7fad280ce730 msgr2=0x7fad280ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 1 -- 10.20.193.6:0/1771828231 shutdown_connections 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 1 -- 10.20.193.6:0/1771828231 wait complete. 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fd78a7fc640 1 -- 10.20.193.6:0/605487740 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370306555 0 0) 0x7fd78c0c8170 con 0x7fd78c0c99a0 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fd78a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fd78a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fd78a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fd78a7fc640 10 monclient(hunting): my global_id is 4535 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fd78a7fc640 1 -- 10.20.193.6:0/605487740 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd78c0c7e90 con 0x7fd78c0c99a0 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fb27b577640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb27c15a5e0 0x7fb27c0bea20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00b970 tx=0x7fb26c00be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fb25b7fe640 10 client.?.objecter ms_handle_connect 0x7fb27c15a5e0 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fb25b7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.158+0000 7fad2fc58640 10 monclient: monmap: 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 librados: starting msgr at 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 librados: starting objecter 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7ff097fff640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff098074d60 0x7ff098142f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b500 tx=0x7ff08400b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7ff094ff9640 10 client.?.objecter ms_handle_connect 0x7ff098074d60 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7ff094ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 Processor -- start 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fda48ac9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fda4415d2c0 0x7fda441f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 -- start start 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 librados: setting wanted keys 2026-02-22T23:23:28.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 librados: calling monclient init 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient: init 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7f59b9190640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59b415d730 0x7f59b41f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fac8a31f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fac8415d730 0x7fac840c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad28073530 0x7fad281f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): picked mon.noname-b con 0x7fad28073530 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad281527f0 0x7fad281e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): picked mon.noname-c con 0x7fad281527f0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad2815f900 0x7fad281f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fda48ac9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fda4415d2c0 0x7fda441f7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36422/0 (socket says 10.20.193.6:36422) 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fda48ac9640 1 -- 10.20.193.6:0/2317141550 learned_addr learned my addr 10.20.193.6:0/2317141550 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): picked mon.noname-a con 0x7fad2815f900 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fda48ac9640 10 monclient(hunting): get_auth_request con 0x7fda4415d2c0 auth_method 0 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fda48ac9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fda48ac9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fda48ac9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fad280741e0 con 0x7fad2815f900 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fad28073f00 con 0x7fad28073530 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad280744c0 con 0x7fad281527f0 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fad2fc58640 10 monclient(hunting): authenticate will time out at 1082.886703s 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fac8a31f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fac8415d730 0x7fac840c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36438/0 (socket says 10.20.193.6:36438) 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fac8a31f640 1 -- 10.20.193.6:0/845472967 learned_addr learned my addr 10.20.193.6:0/845472967 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fac8a31f640 10 monclient(hunting): get_auth_request con 0x7fac8415d730 auth_method 0 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fac8a31f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fac8a31f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.159+0000 7fac8a31f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.160+0000 7fda48ac9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.160+0000 7fda48ac9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.160+0000 7fda48ac9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.160+0000 7fd78b7fe640 1 --2- 10.20.193.6:0/605487740 >> v2:10.20.193.40:3300/0 conn(0x7fd78c06ff20 0x7fd78c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.160+0000 7fac8a31f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.160+0000 7fac8a31f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.160+0000 7fac8a31f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7f59b9190640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59b415d730 0x7f59b41f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58558/0 (socket says 10.20.193.6:58558) 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7f59b9190640 1 -- 10.20.193.6:0/2394814723 learned_addr learned my addr 10.20.193.6:0/2394814723 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7f59b9190640 10 monclient(hunting): get_auth_request con 0x7f59b415d730 auth_method 0 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7f59b9190640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7f59b9190640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7f59b9190640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 10 monclient(hunting): handle_auth_done global_id 4492 payload 824 2026-02-22T23:23:28.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 1 -- 10.20.193.6:0/2317141550 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda441527f0 msgr2=0x7fda440c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda441527f0 0x7fda440c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 1 -- 10.20.193.6:0/2317141550 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fda4415d730 msgr2=0x7fda441f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fda4415d730 0x7fda441f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 1 monclient: found mon.noname-b 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fda48ac9640 1 -- 10.20.193.6:0/2317141550 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda441f4520 con 0x7fda4415d2c0 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fd78b7fe640 10 monclient(hunting): get_auth_request con 0x7fd78c06ff20 auth_method 0 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fd78b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fd78b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.162+0000 7fd78b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 10 monclient(hunting): handle_auth_done global_id 4495 payload 824 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 1 -- 10.20.193.6:0/845472967 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 msgr2=0x7fac841f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 0x7fac841f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 1 -- 10.20.193.6:0/845472967 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fac8415d2c0 msgr2=0x7fac8414ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fac8415d2c0 0x7fac8414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 1 monclient: found mon.noname-b 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.164+0000 7fac8a31f640 1 -- 10.20.193.6:0/845472967 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac840c65f0 con 0x7fac8415d730 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.165+0000 7fd78b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.165+0000 7fd78b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.165+0000 7fd78b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.9 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.165+0000 7fda48ac9640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fda4415d2c0 0x7fda441f7b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda3802f3e0 tx=0x7fda380047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.165+0000 7fda31ffb640 10 client.?.objecter ms_handle_connect 0x7fda4415d2c0 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.165+0000 7fda31ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.166+0000 7fac8a31f640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fac8415d730 0x7fac840c79c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac8000b990 tx=0x7fac8000be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.166+0000 7fac727fc640 10 client.?.objecter ms_handle_connect 0x7fac8415d730 2026-02-22T23:23:28.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.166+0000 7fac727fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 10 monclient(hunting): handle_auth_done global_id 4498 payload 274 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 1 -- 10.20.193.6:0/605487740 >> v1:10.20.193.80:6789/0 conn(0x7fd78c0c99a0 legacy=0x7fd78c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 1 -- 10.20.193.6:0/605487740 >> v1:10.20.193.6:6789/0 conn(0x7fd78c15a5e0 legacy=0x7fd78c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 1 monclient: found mon.noname-b 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7fd78b7fe640 1 -- 10.20.193.6:0/605487740 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c0c8450 con 0x7fd78c06ff20 2026-02-22T23:23:28.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.167+0000 7f99ecff9640 1 -- 10.20.193.6:0/1558078321 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f99e4010480 con 0x7f99f0073530 2026-02-22T23:23:28.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.168+0000 7f99ecff9640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f99c0078e50 0x7f99c007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.168+0000 7f99ef7fe640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f99c0078e50 0x7f99c007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.168+0000 7f99ef7fe640 10 monclient: get_auth_request con 0x7f99c0078e50 auth_method 0 2026-02-22T23:23:28.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7fd78b7fe640 1 --2- 10.20.193.6:0/605487740 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 0x7fd78c070300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7780098d0 tx=0x7fd77802eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ced217794d7789 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7f99ef7fe640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f99c0078e50 0x7f99c007b310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99f01f6c50 tx=0x7f99d8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7f99ecff9640 10 client.4523.objecter ms_handle_connect 0x7f99c0078e50 2026-02-22T23:23:28.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7f99ecff9640 1 -- 10.20.193.6:0/1558078321 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f99e405e220 con 0x7f99f0073530 2026-02-22T23:23:28.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7f99ecff9640 10 client.4523.objecter ms_dispatch2 0x7f99f0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7f99ecff9640 3 client.4523.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7f99ecff9640 3 client.4523.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.169+0000 7f99ecff9640 20 client.4523.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.170+0000 7f59b9190640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.170+0000 7f59b9190640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.171+0000 7f59b9190640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.173+0000 7f59abfff640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59b415d2c0 0x7f59b41f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.174+0000 7f59abfff640 10 monclient(hunting): get_auth_request con 0x7f59b415d2c0 auth_method 0 2026-02-22T23:23:28.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.174+0000 7f59abfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.174+0000 7f59abfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.174+0000 7f59abfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.174+0000 7f59abfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.174+0000 7f59abfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.174+0000 7f59abfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 1 -- 10.20.193.6:0/2317141550 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38044070 con 0x7fda4415d2c0 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: dump: 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 1 -- 10.20.193.6:0/2317141550 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda38040630 con 0x7fda4415d2c0 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 1 -- 10.20.193.6:0/2317141550 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3803c040 con 0x7fda4415d2c0 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 5 monclient: authenticate success, global_id 4492 2026-02-22T23:23:28.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda337fe640 4 set_mon_vals no callback set 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda31ffb640 10 monclient: dump: 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 monclient: _renew_subs 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda441f4800 con 0x7fda4415d2c0 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 client.4492.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 monclient: _renew_subs 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda440c58d0 con 0x7fda4415d2c0 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 1 librados: init done 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda440734b0 con 0x7fda4415d2c0 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 1 -- 10.20.193.6:0/845472967 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8000c790 con 0x7fac8415d730 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: dump: 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 1 -- 10.20.193.6:0/845472967 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac80010070 con 0x7fac8415d730 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 1 -- 10.20.193.6:0/845472967 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8000cbb0 con 0x7fac8415d730 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 5 monclient: authenticate success, global_id 4495 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 10 monclient: _renew_subs 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac73fff640 4 set_mon_vals no callback set 2026-02-22T23:23:28.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac840c6c00 con 0x7fac8415d730 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac727fc640 10 monclient: dump: 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 10 client.4495.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 10 monclient: _renew_subs 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac840c71c0 con 0x7fac8415d730 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.175+0000 7fac8bda9640 1 librados: init done 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.176+0000 7fac8bda9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.176+0000 7fac8bda9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.176+0000 7fac8bda9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.176+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac84152bd0 con 0x7fac8415d730 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 10 monclient(hunting): handle_auth_done global_id 4501 payload 824 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 1 -- 10.20.193.6:0/2394814723 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59b415d730 msgr2=0x7f59b41f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59b415d730 0x7f59b41f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 1 -- 10.20.193.6:0/2394814723 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b41527f0 msgr2=0x7f59b414dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b41527f0 0x7f59b414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 1 monclient: found mon.noname-b 2026-02-22T23:23:28.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f59abfff640 1 -- 10.20.193.6:0/2394814723 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b414c830 con 0x7f59b415d2c0 2026-02-22T23:23:28.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f99ecff9640 1 -- 10.20.193.6:0/1558078321 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f99e400d360 con 0x7f99f0073530 2026-02-22T23:23:28.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f99ecff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.177+0000 7f99ecff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.180+0000 7f7d8621e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d801527f0 0x7f7d801f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.182+0000 7f59abfff640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59b415d2c0 0x7f59b41f5190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f599c006fd0 tx=0x7f599c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.182+0000 7f59a97fa640 10 client.?.objecter ms_handle_connect 0x7f59b415d2c0 2026-02-22T23:23:28.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.182+0000 7f59a97fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7f7d8621e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d801527f0 0x7f7d801f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58564/0 (socket says 10.20.193.6:58564) 2026-02-22T23:23:28.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7f7d8621e640 1 -- 10.20.193.6:0/1949183421 learned_addr learned my addr 10.20.193.6:0/1949183421 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7f7d8621e640 10 monclient(hunting): get_auth_request con 0x7f7d801527f0 auth_method 0 2026-02-22T23:23:28.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7f7d8621e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7f7d8621e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7f7d8621e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 1 -- 10.20.193.6:0/88144848 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c002c70 con 0x7fb27c15a5e0 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: dump: 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 1 -- 10.20.193.6:0/88144848 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26c010070 con 0x7fb27c15a5e0 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 1 -- 10.20.193.6:0/88144848 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c00c9d0 con 0x7fb27c15a5e0 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb25b7fe640 10 monclient: dump: 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb279573640 4 set_mon_vals no callback set 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 5 monclient: authenticate success, global_id 4538 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 10 monclient: _renew_subs 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c0bdc60 con 0x7fb27c15a5e0 2026-02-22T23:23:28.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 10 client.4538.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 10 monclient: _renew_subs 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c0be220 con 0x7fb27c15a5e0 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.185+0000 7fb28102b640 1 librados: init done 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.186+0000 7f7d8621e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.186+0000 7f7d8621e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.186+0000 7f7d8621e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.186+0000 7fb28102b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.186+0000 7fb28102b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.186+0000 7fb28102b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.186+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27c070300 con 0x7fb27c15a5e0 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 10 monclient(hunting): handle_auth_done global_id 4547 payload 824 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 1 -- 10.20.193.6:0/1949183421 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d8015d2c0 msgr2=0x7f7d8014ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d8015d2c0 0x7f7d8014ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 1 -- 10.20.193.6:0/1949183421 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d8015d730 msgr2=0x7f7d800c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d8015d730 0x7f7d800c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.188+0000 7f7d8621e640 1 -- 10.20.193.6:0/1949183421 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d800c65f0 con 0x7f7d801527f0 2026-02-22T23:23:28.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.190+0000 7fda31ffb640 1 -- 10.20.193.6:0/2317141550 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fda3803f050 con 0x7fda4415d2c0 2026-02-22T23:23:28.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.190+0000 7fda31ffb640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fda14078e50 0x7fda1407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.190+0000 7fda492ca640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fda14078e50 0x7fda1407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.190+0000 7fda492ca640 10 monclient: get_auth_request con 0x7fda14078e50 auth_method 0 2026-02-22T23:23:28.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.191+0000 7fda492ca640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fda14078e50 0x7fda1407b310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda340098a0 tx=0x7fda34006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.191+0000 7fda31ffb640 10 client.4492.objecter ms_handle_connect 0x7fda14078e50 2026-02-22T23:23:28.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.192+0000 7fda31ffb640 1 -- 10.20.193.6:0/2317141550 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fda38082f00 con 0x7fda4415d2c0 2026-02-22T23:23:28.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.192+0000 7fda31ffb640 10 client.4492.objecter ms_dispatch2 0x7fda44000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.192+0000 7fda31ffb640 3 client.4492.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.192+0000 7fda31ffb640 3 client.4492.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.192+0000 7fda31ffb640 20 client.4492.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.193+0000 7f5aa1ffb640 1 -- 10.20.193.6:0/3039883624 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f5ab000ae20 con 0x7f5ab415d730 2026-02-22T23:23:28.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.193+0000 7f5aa1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-22T23:23:28.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.193+0000 7f5aa1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.194 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:28.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 10 librados: watch_flush enter 2026-02-22T23:23:28.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 10 librados: watch_flush exit 2026-02-22T23:23:28.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 20 client.4514.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 10 client.4514.objecter successfully canceled tick 2026-02-22T23:23:28.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5a84078e50 msgr2=0x7f5a8407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5a84078e50 0x7f5a8407b310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006b90 tx=0x7f5aa4006b20 comp rx=0 tx=0).stop 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 10 monclient: shutdown 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d730 msgr2=0x7f5ab415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d730 0x7f5ab415a650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000cad0 tx=0x7f5ab000a060 comp rx=0 tx=0).stop 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 shutdown_connections 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5a84078e50 0x7f5a8407b310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5ab415d730 0x7f5ab415a650 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5ab415d2c0 0x7f5ab42042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 --2- 10.20.193.6:0/3039883624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5ab41527f0 0x7f5ab415c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 >> 10.20.193.6:0/3039883624 conn(0x7f5ab40ce730 msgr2=0x7f5ab40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 shutdown_connections 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 -- 10.20.193.6:0/3039883624 wait complete. 2026-02-22T23:23:28.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.195+0000 7f5abb3eb640 1 librados: shutdown 2026-02-22T23:23:28.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.14 2026-02-22T23:23:28.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.235+0000 7f463712c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:28.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.235+0000 7f463712c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:28.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.235+0000 7f463712c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.235+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f46301597f0 con 0x7f463015d2c0 2026-02-22T23:23:28.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 1 -- 10.20.193.6:0/3351743640 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4017070 con 0x7f4aac004900 2026-02-22T23:23:28.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: dump: 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 1 -- 10.20.193.6:0/3351743640 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab4005de0 con 0x7f4aac004900 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 1 -- 10.20.193.6:0/3351743640 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4007270 con 0x7f4aac004900 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 5 monclient: authenticate success, global_id 4482 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 10 monclient: _renew_subs 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab3fff640 4 set_mon_vals no callback set 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac41501b0 con 0x7f4aac004900 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4ab27fc640 10 monclient: dump: 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 10 client.4482.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 10 monclient: _renew_subs 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac4150770 con 0x7f4aac004900 2026-02-22T23:23:28.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.259+0000 7f4acbde4640 1 librados: init done 2026-02-22T23:23:28.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.260+0000 7f4acbde4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.260+0000 7f4acbde4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.260+0000 7f4acbde4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.260+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac41509b0 con 0x7f4aac004900 2026-02-22T23:23:28.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.263+0000 7f99f68b0640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:28.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.263+0000 7f99f68b0640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:28.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.263+0000 7f99f68b0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.263+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f99f015dd40 con 0x7f99f0073530 2026-02-22T23:23:28.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.267+0000 7f76f67fc640 1 -- 10.20.193.6:0/738428046 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f77040050d0 con 0x7f770815f650 2026-02-22T23:23:28.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.268+0000 7f76f67fc640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f76d8078e50 0x7f76d807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.268+0000 7f770d871640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f76d8078e50 0x7f76d807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.268+0000 7f770d871640 10 monclient: get_auth_request con 0x7f76d8078e50 auth_method 0 2026-02-22T23:23:28.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.268+0000 7f76f67fc640 10 client.4461.objecter ms_handle_connect 0x7f76d8078e50 2026-02-22T23:23:28.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.268+0000 7f770d871640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f76d8078e50 0x7f76d807b310 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77080cb8c0 tx=0x7f76f8031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 1 -- 10.20.193.6:0/738428046 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f770405e7a0 con 0x7f770815f650 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 10 client.4461.objecter ms_dispatch2 0x7f7708000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 3 client.4461.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 3 client.4461.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 20 client.4461.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 1 -- 10.20.193.6:0/738428046 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7704019300 con 0x7f770815f650 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.278+0000 7f76f67fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.285+0000 7f5877fff640 1 -- 10.20.193.6:0/2784236733 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5890087b10 con 0x7f589c14bdb0 2026-02-22T23:23:28.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.285+0000 7f5877fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-22T23:23:28.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.285+0000 7f5877fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.286 INFO:teuthology.orchestra.run.trial006.stdout:94489280514 2026-02-22T23:23:28.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.286+0000 7f7d8621e640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d801527f0 0x7f7d801f41c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009aa0 tx=0x7f7d700047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.286+0000 7f7d66ffd640 10 client.?.objecter ms_handle_connect 0x7f7d801527f0 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.286+0000 7f7d66ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.286+0000 7fad2d1cc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad281527f0 0x7fad281e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad281527f0 0x7fad281e81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58568/0 (socket says 10.20.193.6:58568) 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1574591571 learned_addr learned my addr 10.20.193.6:0/1574591571 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 10 monclient(hunting): get_auth_request con 0x7fad281527f0 auth_method 0 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 10 librados: watch_flush enter 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 10 librados: watch_flush exit 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 20 client.4449.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 10 client.4449.objecter successfully canceled tick 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f586c078e50 msgr2=0x7f586c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f586c078e50 0x7f586c07b310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5888011310 tx=0x7f5888009290 comp rx=0 tx=0).stop 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 10 monclient: shutdown 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f589c14bdb0 msgr2=0x7f589c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f589c14bdb0 0x7f589c070600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5890039040 tx=0x7f5890004b40 comp rx=0 tx=0).stop 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 shutdown_connections 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f586c078e50 0x7f586c07b310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f589c14bdb0 0x7f589c070600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f589c155f80 0x7f589c14b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 --2- 10.20.193.6:0/2784236733 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f589c0ccb40 0x7f589c1faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 >> 10.20.193.6:0/2784236733 conn(0x7f589c0cc020 msgr2=0x7f589c0cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 shutdown_connections 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 -- 10.20.193.6:0/2784236733 wait complete. 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7f58a1dd6640 1 librados: shutdown 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.287+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 10 monclient(hunting): handle_auth_done global_id 4550 payload 824 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1574591571 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad28073530 msgr2=0x7fad281f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad28073530 0x7fad281f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1574591571 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad2815f900 msgr2=0x7fad281f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad2815f900 0x7fad281f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 1 monclient: found mon.noname-c 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.289+0000 7fad2d1cc640 1 -- 10.20.193.6:0/1574591571 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad281f7e60 con 0x7fad281527f0 2026-02-22T23:23:28.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.293+0000 7fad0affd640 1 -- 10.20.193.6:0/2911049934 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fad18002a70 con 0x7fad10004900 2026-02-22T23:23:28.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.294+0000 7fad0affd640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf0078e80 0x7facf007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.294+0000 7fad21d74640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf0078e80 0x7facf007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.294+0000 7fad21d74640 10 monclient: get_auth_request con 0x7facf0078e80 auth_method 0 2026-02-22T23:23:28.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.294+0000 7fad21d74640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf0078e80 0x7facf007b340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad0c036000 tx=0x7fad0c009070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.294+0000 7fad0affd640 10 client.4479.objecter ms_handle_connect 0x7facf0078e80 2026-02-22T23:23:28.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280514 for osd.1 2026-02-22T23:23:28.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.296+0000 7fb25b7fe640 1 -- 10.20.193.6:0/88144848 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb26c01d030 con 0x7fb27c15a5e0 2026-02-22T23:23:28.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.296+0000 7fb25b7fe640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb248078e50 0x7fb24807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.297+0000 7fb27ad76640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb248078e50 0x7fb24807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.297+0000 7fb27ad76640 10 monclient: get_auth_request con 0x7fb248078e50 auth_method 0 2026-02-22T23:23:28.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.297+0000 7fb27ad76640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb248078e50 0x7fb24807b310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb270039000 tx=0x7fb270033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.297+0000 7fb25b7fe640 10 client.4538.objecter ms_handle_connect 0x7fb248078e50 2026-02-22T23:23:28.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.298+0000 7fb25b7fe640 1 -- 10.20.193.6:0/88144848 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fb26c05d8b0 con 0x7fb27c15a5e0 2026-02-22T23:23:28.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.298+0000 7fb25b7fe640 10 client.4538.objecter ms_dispatch2 0x7fb27c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.298+0000 7fb25b7fe640 3 client.4538.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.298+0000 7fb25b7fe640 3 client.4538.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.298+0000 7fb25b7fe640 20 client.4538.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.301+0000 7fac727fc640 1 -- 10.20.193.6:0/845472967 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fac8001d030 con 0x7fac8415d730 2026-02-22T23:23:28.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.301+0000 7fac727fc640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fac54078e00 0x7fac5407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.302+0000 7fac89b1e640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fac54078e00 0x7fac5407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.302+0000 7fac89b1e640 10 monclient: get_auth_request con 0x7fac54078e00 auth_method 0 2026-02-22T23:23:28.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.302+0000 7fac727fc640 10 client.4495.objecter ms_handle_connect 0x7fac54078e00 2026-02-22T23:23:28.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.302+0000 7fac89b1e640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fac54078e00 0x7fac5407b2c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fac8406cdf0 tx=0x7fac74031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 1 -- 10.20.193.6:0/2911049934 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fad18002d90 con 0x7fad10004900 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 10 client.4479.objecter ms_dispatch2 0x7fad24000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 3 client.4479.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 3 client.4479.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 20 client.4479.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 1 -- 10.20.193.6:0/2911049934 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fad1801d020 con 0x7fad10004900 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.303+0000 7fad0affd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.305+0000 7fb25b7fe640 1 -- 10.20.193.6:0/88144848 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb26c024080 con 0x7fb27c15a5e0 2026-02-22T23:23:28.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.305+0000 7fb25b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.305+0000 7fb25b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.306+0000 7fad2d1cc640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad281527f0 0x7fad281e81d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009970 tx=0x7fad1c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.306+0000 7fad167fc640 10 client.?.objecter ms_handle_connect 0x7fad281527f0 2026-02-22T23:23:28.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.306+0000 7fad167fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 1 -- 10.20.193.6:0/2089095073 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084004480 con 0x7ff098074d60 2026-02-22T23:23:28.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: dump: 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.311 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 1 -- 10.20.193.6:0/2089095073 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff084014070 con 0x7ff098074d60 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 1 -- 10.20.193.6:0/2089095073 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084002920 con 0x7ff098074d60 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff094ff9640 10 monclient: dump: 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.312 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff0967fc640 4 set_mon_vals no callback set 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 5 monclient: authenticate success, global_id 4541 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 10 monclient: _renew_subs 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff098143960 con 0x7ff098074d60 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 10 client.4541.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 10 monclient: _renew_subs 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff098143f20 con 0x7ff098074d60 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.308+0000 7ff09e340640 1 librados: init done 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.309+0000 7ff09e340640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.309+0000 7ff09e340640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.309+0000 7ff09e340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.309+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff098074120 con 0x7ff098074d60 2026-02-22T23:23:28.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fac727fc640 1 -- 10.20.193.6:0/845472967 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fac8000cd50 con 0x7fac8415d730 2026-02-22T23:23:28.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fac727fc640 10 client.4495.objecter ms_dispatch2 0x7fac84000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fac727fc640 3 client.4495.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fac727fc640 3 client.4495.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fac727fc640 20 client.4495.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fda31ffb640 1 -- 10.20.193.6:0/2317141550 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fda380875c0 con 0x7fda4415d2c0 2026-02-22T23:23:28.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fda31ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.310+0000 7fda31ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 1 -- 10.20.193.6:0/1574591571 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c044070 con 0x7fad281527f0 2026-02-22T23:23:28.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: dump: 2026-02-22T23:23:28.317 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 1 -- 10.20.193.6:0/1574591571 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1c040610 con 0x7fad281527f0 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 1 -- 10.20.193.6:0/1574591571 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c03c040 con 0x7fad281527f0 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad167fc640 10 monclient: dump: 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.319 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad17fff640 4 set_mon_vals no callback set 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 5 monclient: authenticate success, global_id 4550 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 monclient: _renew_subs 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2814bbd0 con 0x7fad281527f0 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 client.4550.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 monclient: _renew_subs 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281e7410 con 0x7fad281527f0 2026-02-22T23:23:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 1 librados: init done 2026-02-22T23:23:28.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 1 -- 10.20.193.6:0/1949183421 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70044070 con 0x7f7d801527f0 2026-02-22T23:23:28.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: dump: 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 1 -- 10.20.193.6:0/1949183421 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d70040610 con 0x7f7d801527f0 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 1 -- 10.20.193.6:0/1949183421 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7003c040 con 0x7f7d801527f0 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 5 monclient: authenticate success, global_id 4547 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 10 monclient: _renew_subs 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d84a1b640 4 set_mon_vals no callback set 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d800c6c00 con 0x7f7d801527f0 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d66ffd640 10 monclient: dump: 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.323 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 10 client.4547.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 10 monclient: _renew_subs 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d800c71c0 con 0x7f7d801527f0 2026-02-22T23:23:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7f7d884a9640 1 librados: init done 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.312+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad280747a0 con 0x7fad281527f0 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.313+0000 7f7d884a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.313+0000 7f7d884a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.313+0000 7f7d884a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.313+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d80152bd0 con 0x7f7d801527f0 2026-02-22T23:23:28.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.313+0000 7f46257fa640 1 -- 10.20.193.6:0/1868574122 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46180620f0 con 0x7f463015d2c0 2026-02-22T23:23:28.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.313+0000 7f46257fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:28.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.313+0000 7f46257fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.326 INFO:teuthology.orchestra.run.trial006.stdout:94489280515 2026-02-22T23:23:28.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 10 librados: watch_flush enter 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 10 librados: watch_flush exit 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 20 client.4486.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 10 client.4486.objecter successfully canceled tick 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4600078e00 msgr2=0x7f460007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4600078e00 0x7f460007b2c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f46200098a0 tx=0x7f4620006d90 comp rx=0 tx=0).stop 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 10 monclient: shutdown 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d2c0 msgr2=0x7f463015fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d2c0 0x7f463015fea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f461800c5b0 tx=0x7f461800ca80 comp rx=0 tx=0).stop 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 shutdown_connections 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4600078e00 0x7f460007b2c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f463015d730 0x7f4630148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f463015d2c0 0x7f463015fea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 --2- 10.20.193.6:0/1868574122 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f46301527f0 0x7f46301f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 >> 10.20.193.6:0/1868574122 conn(0x7f46300ce730 msgr2=0x7f46300d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 shutdown_connections 2026-02-22T23:23:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 -- 10.20.193.6:0/1868574122 wait complete. 2026-02-22T23:23:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.315+0000 7f463712c640 1 librados: shutdown 2026-02-22T23:23:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.317+0000 7f99ecff9640 1 -- 10.20.193.6:0/1558078321 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f99e40628e0 con 0x7f99f0073530 2026-02-22T23:23:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.317+0000 7f99ecff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.317+0000 7f99ecff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.329 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.319+0000 7f99f68b0640 10 librados: watch_flush enter 2026-02-22T23:23:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.319+0000 7f99f68b0640 10 librados: watch_flush exit 2026-02-22T23:23:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.319+0000 7f99f68b0640 20 client.4523.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.319+0000 7f99f68b0640 10 client.4523.objecter successfully canceled tick 2026-02-22T23:23:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f99c0078e50 msgr2=0x7f99c007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f99c0078e50 0x7f99c007b310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99f01f6c50 tx=0x7f99d8008040 comp rx=0 tx=0).stop 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 10 monclient: shutdown 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f99f0073530 msgr2=0x7f99f015e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f99f0073530 0x7f99f015e4d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99e400b570 tx=0x7f99e400ba40 comp rx=0 tx=0).stop 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 shutdown_connections 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f99c0078e50 0x7f99c007b310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f99f015fad0 0x7f99f01f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f99f01527f0 0x7f99f01f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 --2- 10.20.193.6:0/1558078321 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f99f0073530 0x7f99f015e4d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 >> 10.20.193.6:0/1558078321 conn(0x7f99f00ce730 msgr2=0x7f99f00ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 shutdown_connections 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 -- 10.20.193.6:0/1558078321 wait complete. 2026-02-22T23:23:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.320+0000 7f99f68b0640 1 librados: shutdown 2026-02-22T23:23:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.324+0000 7fad167fc640 1 -- 10.20.193.6:0/1574591571 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fad1c03f050 con 0x7fad281527f0 2026-02-22T23:23:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.325+0000 7fad167fc640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf8078e50 0x7facf807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280515 for osd.8 2026-02-22T23:23:28.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.325+0000 7fad2d9cd640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf8078e50 0x7facf807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.325+0000 7fad2d9cd640 10 monclient: get_auth_request con 0x7facf8078e50 auth_method 0 2026-02-22T23:23:28.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.326+0000 7fad167fc640 10 client.4550.objecter ms_handle_connect 0x7facf8078e50 2026-02-22T23:23:28.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.326+0000 7fad2d9cd640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf8078e50 0x7facf807b310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad18006fd0 tx=0x7fad18006d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.7 2026-02-22T23:23:28.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.366+0000 7f770fafc640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:28.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.366+0000 7f770fafc640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:28.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.366+0000 7f770fafc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.366+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7708151510 con 0x7f770815f650 2026-02-22T23:23:28.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fad28c61640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:28.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fad28c61640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:28.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fad28c61640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fad2415ec50 con 0x7fad10004900 2026-02-22T23:23:28.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fb28102b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:28.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fb28102b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:28.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fb28102b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.392+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7fb27c1573f0 con 0x7fb27c15a5e0 2026-02-22T23:23:28.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.398+0000 7fda4b555640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:28.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.398+0000 7fda4b555640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:28.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.398+0000 7fda4b555640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.398+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7fda44152bd0 con 0x7fda4415d2c0 2026-02-22T23:23:28.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.421+0000 7f4ab27fc640 1 -- 10.20.193.6:0/3351743640 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4ab4007410 con 0x7f4aac004900 2026-02-22T23:23:28.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7f4ab27fc640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a94078e50 0x7f4a9407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7f4ac9358640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a94078e50 0x7f4a9407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7f4ac9358640 10 monclient: get_auth_request con 0x7f4a94078e50 auth_method 0 2026-02-22T23:23:28.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7fac727fc640 1 -- 10.20.193.6:0/845472967 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fac80062110 con 0x7fac8415d730 2026-02-22T23:23:28.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7fac727fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7fac727fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7f4ab27fc640 10 client.4482.objecter ms_handle_connect 0x7f4a94078e50 2026-02-22T23:23:28.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.422+0000 7f4ac9358640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a94078e50 0x7f4a9407b310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4ab803b000 tx=0x7f4ab8035000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 1 -- 10.20.193.6:0/605487740 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77803c070 con 0x7fd78c06ff20 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: dump: 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.426 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 1 -- 10.20.193.6:0/605487740 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77802fba0 con 0x7fd78c06ff20 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 1 -- 10.20.193.6:0/605487740 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778037440 con 0x7fd78c06ff20 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:28.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd78a7fc640 10 monclient: dump: 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd7921d3640 5 monclient: authenticate success, global_id 4498 2026-02-22T23:23:28.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd7921d3640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:28.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd7921d3640 4 set_mon_vals no callback set 2026-02-22T23:23:28.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fda31ffb640 1 -- 10.20.193.6:0/2317141550 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fda38086ec0 con 0x7fda4415d2c0 2026-02-22T23:23:28.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fda31ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:28.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fda31ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.429 INFO:teuthology.orchestra.run.trial006.stdout:94489280515 2026-02-22T23:23:28.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd7921d3640 10 monclient: shutdown 2026-02-22T23:23:28.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd7921d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd7921d3640 1 -- 10.20.193.6:0/605487740 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 msgr2=0x7fd78c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.424+0000 7fd7921d3640 1 --2- 10.20.193.6:0/605487740 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 0x7fd78c070300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7780098d0 tx=0x7fd77802eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 -- 10.20.193.6:0/605487740 shutdown_connections 2026-02-22T23:23:28.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 --2- 10.20.193.6:0/605487740 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 0x7fd78c070300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 -- 10.20.193.6:0/605487740 >> 10.20.193.6:0/605487740 conn(0x7fd78c0cd780 msgr2=0x7fd78c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 -- 10.20.193.6:0/605487740 shutdown_connections 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 -- 10.20.193.6:0/605487740 wait complete. 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 10 monclient: build_initial_monmap 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 10 monclient: monmap: 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:28.432 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 librados: starting msgr at 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 librados: starting objecter 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 Processor -- start 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 -- start start 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 librados: setting wanted keys 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 1 librados: calling monclient init 2026-02-22T23:23:28.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.425+0000 7fd7921d3640 10 monclient: init 2026-02-22T23:23:28.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 1 -- 10.20.193.6:0/2394814723 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c044070 con 0x7f59b415d2c0 2026-02-22T23:23:28.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:28.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: dump: 2026-02-22T23:23:28.434 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 1 -- 10.20.193.6:0/2394814723 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c040610 con 0x7f59b415d2c0 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 1 -- 10.20.193.6:0/2394814723 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c03c040 con 0x7f59b415d2c0 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59a97fa640 10 monclient: dump: 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.436 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 5 monclient: authenticate success, global_id 4501 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59aaffd640 4 set_mon_vals no callback set 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 monclient: _renew_subs 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b414cb10 con 0x7f59b415d2c0 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 client.4501.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 monclient: _renew_subs 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b414d0d0 con 0x7f59b415d2c0 2026-02-22T23:23:28.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 1 librados: init done 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 0x7fd78c14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): picked mon.noname-b con 0x7fd78c06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd78c0c99a0 0x7fd78c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): picked mon.noname-a con 0x7fd78c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd78c15a5e0 0x7fd78c155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): picked mon.noname-c con 0x7fd78c15a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c150520 con 0x7fd78c0c99a0 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c0c8a10 con 0x7fd78c06ff20 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1506a0 con 0x7fd78c15a5e0 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:28.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fd7921d3640 10 monclient(hunting): authenticate will time out at 1083.153253s 2026-02-22T23:23:28.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b40cbee0 con 0x7f59b415d2c0 2026-02-22T23:23:28.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fda4b555640 10 librados: watch_flush enter 2026-02-22T23:23:28.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fda4b555640 10 librados: watch_flush exit 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fda4b555640 20 client.4492.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.426+0000 7fda4b555640 10 client.4492.objecter successfully canceled tick 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fda14078e50 msgr2=0x7fda1407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fda14078e50 0x7fda1407b310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fda340098a0 tx=0x7fda34006d90 comp rx=0 tx=0).stop 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 10 monclient: shutdown 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fda4415d2c0 msgr2=0x7fda441f7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fda4415d2c0 0x7fda441f7b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda3802f3e0 tx=0x7fda380047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 shutdown_connections 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fda14078e50 0x7fda1407b310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fda4415d730 0x7fda441f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fda4415d2c0 0x7fda441f7b60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 --2- 10.20.193.6:0/2317141550 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fda441527f0 0x7fda440c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 >> 10.20.193.6:0/2317141550 conn(0x7fda440ce730 msgr2=0x7fda440d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 shutdown_connections 2026-02-22T23:23:28.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 -- 10.20.193.6:0/2317141550 wait complete. 2026-02-22T23:23:28.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.427+0000 7fda4b555640 1 librados: shutdown 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 1 -- 10.20.193.6:0/3351743640 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f4ab4007730 con 0x7f4aac004900 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 10 client.4482.objecter ms_dispatch2 0x7f4ac4000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 3 client.4482.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 3 client.4482.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 20 client.4482.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 1 -- 10.20.193.6:0/3351743640 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4ac41509b0 con 0x7f4aac004900 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.430+0000 7f4ab27fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.431+0000 7fd78b7fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 0x7fd78c14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.434+0000 7f76f67fc640 1 -- 10.20.193.6:0/738428046 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7704062e60 con 0x7f770815f650 2026-02-22T23:23:28.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.434+0000 7f76f67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:28.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.434+0000 7f76f67fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.447 INFO:teuthology.orchestra.run.trial006.stdout:94489280515 2026-02-22T23:23:28.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.434+0000 7fad0affd640 1 -- 10.20.193.6:0/2911049934 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fad180625f0 con 0x7fad10004900 2026-02-22T23:23:28.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.434+0000 7fad0affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:28.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.434+0000 7fad0affd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.448 INFO:teuthology.orchestra.run.trial006.stdout:98784247810 2026-02-22T23:23:28.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.435+0000 7fd78affd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd78c0c99a0 0x7fd78c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.435+0000 7fd78affd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd78c0c99a0 0x7fd78c154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39802/0 (socket says 10.20.193.6:39802) 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.435+0000 7fd78affd640 1 -- 10.20.193.6:0/2388511178 learned_addr learned my addr 10.20.193.6:0/2388511178 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.435+0000 7fd78affd640 10 monclient(hunting): get_auth_request con 0x7fd78c0c99a0 auth_method 0 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.435+0000 7fd78affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.435+0000 7fd78affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.435+0000 7fd78affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.436+0000 7fd78affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.436+0000 7fd78affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:28.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.436+0000 7fd78affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:28.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280515 for osd.13 2026-02-22T23:23:28.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 10 librados: watch_flush enter 2026-02-22T23:23:28.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 10 librados: watch_flush exit 2026-02-22T23:23:28.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 20 client.4461.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 10 client.4461.objecter successfully canceled tick 2026-02-22T23:23:28.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f76d8078e50 msgr2=0x7f76d807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f76d8078e50 0x7f76d807b310 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77080cb8c0 tx=0x7f76f8031000 comp rx=0 tx=0).stop 2026-02-22T23:23:28.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 10 monclient: shutdown 2026-02-22T23:23:28.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f770815f650 msgr2=0x7f77081e9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f770815f650 0x7f77081e9340 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f770400cb70 tx=0x7f770400a430 comp rx=0 tx=0).stop 2026-02-22T23:23:28.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 10 librados: watch_flush enter 2026-02-22T23:23:28.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 10 librados: watch_flush exit 2026-02-22T23:23:28.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 shutdown_connections 2026-02-22T23:23:28.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f76d8078e50 0x7f76d807b310 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f770815f650 0x7f77081e9340 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 20 client.4479.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 10 client.4479.objecter successfully canceled tick 2026-02-22T23:23:28.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f77080d1ea0 0x7f7708158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 --2- 10.20.193.6:0/738428046 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f77080ceca0 0x7f77081f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 >> 10.20.193.6:0/738428046 conn(0x7f7708072fc0 msgr2=0x7f77080c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 shutdown_connections 2026-02-22T23:23:28.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf0078e80 msgr2=0x7facf007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf0078e80 0x7facf007b340 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad0c036000 tx=0x7fad0c009070 comp rx=0 tx=0).stop 2026-02-22T23:23:28.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 10 monclient: shutdown 2026-02-22T23:23:28.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10004900 msgr2=0x7fad24150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 -- 10.20.193.6:0/738428046 wait complete. 2026-02-22T23:23:28.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10004900 0x7fad24150a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad1800b5a0 tx=0x7fad1800ba70 comp rx=0 tx=0).stop 2026-02-22T23:23:28.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7f770fafc640 1 librados: shutdown 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 shutdown_connections 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf0078e80 0x7facf007b340 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad24159c40 0x7fad240c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad24150f80 0x7fad240c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 --2- 10.20.193.6:0/2911049934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad10004900 0x7fad24150a40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 >> 10.20.193.6:0/2911049934 conn(0x7fad240ce730 msgr2=0x7fad2415e020 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 shutdown_connections 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 -- 10.20.193.6:0/2911049934 wait complete. 2026-02-22T23:23:28.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fad28c61640 1 librados: shutdown 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 10 monclient(hunting): handle_auth_done global_id 4485 payload 824 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 1 -- 10.20.193.6:0/2388511178 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd78c15a5e0 msgr2=0x7fd78c155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd78c15a5e0 0x7fd78c155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 1 -- 10.20.193.6:0/2388511178 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 msgr2=0x7fd78c14bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 0x7fd78c14bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 1 monclient: found mon.noname-a 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.437+0000 7fd78affd640 1 -- 10.20.193.6:0/2388511178 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c156d60 con 0x7fd78c0c99a0 2026-02-22T23:23:28.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.438+0000 7f59a97fa640 1 -- 10.20.193.6:0/2394814723 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f599c03f050 con 0x7f59b415d2c0 2026-02-22T23:23:28.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.438+0000 7f59a97fa640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5984078e50 0x7f598407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.439+0000 7f59b898f640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5984078e50 0x7f598407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.439+0000 7f59b898f640 10 monclient: get_auth_request con 0x7f5984078e50 auth_method 0 2026-02-22T23:23:28.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.439+0000 7fd78affd640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd78c0c99a0 0x7fd78c154f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00c700 tx=0x7fd77c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.439+0000 7fd767fff640 10 client.?.objecter ms_handle_connect 0x7fd78c0c99a0 2026-02-22T23:23:28.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.439+0000 7fd767fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:28.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.439+0000 7f59b898f640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5984078e50 0x7f598407b310 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f59a40098a0 tx=0x7f59a4006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.439+0000 7f59a97fa640 10 client.4501.objecter ms_handle_connect 0x7f5984078e50 2026-02-22T23:23:28.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.440+0000 7f59a97fa640 1 -- 10.20.193.6:0/2394814723 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f599c082e00 con 0x7f59b415d2c0 2026-02-22T23:23:28.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.440+0000 7f59a97fa640 10 client.4501.objecter ms_dispatch2 0x7f59b4000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.440+0000 7f59a97fa640 3 client.4501.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.440+0000 7f59a97fa640 3 client.4501.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.440+0000 7f59a97fa640 20 client.4501.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 1 -- 10.20.193.6:0/2388511178 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0052d0 con 0x7fd78c0c99a0 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: dump: 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:28.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 1 -- 10.20.193.6:0/2388511178 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77c005490 con 0x7fd78c0c99a0 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 1 -- 10.20.193.6:0/2388511178 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0038d0 con 0x7fd78c0c99a0 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 5 monclient: authenticate success, global_id 4485 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd789ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd767fff640 10 monclient: dump: 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:28.463 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 10 monclient: _renew_subs 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c157040 con 0x7fd78c0c99a0 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 10 client.4485.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 10 monclient: _renew_subs 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c154720 con 0x7fd78c0c99a0 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.441+0000 7fd7921d3640 1 librados: init done 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.442+0000 7fd7921d3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.442+0000 7fd7921d3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.442+0000 7fd7921d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.442+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78c070300 con 0x7fd78c0c99a0 2026-02-22T23:23:28.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247810 for osd.0 2026-02-22T23:23:28.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280515 for osd.4 2026-02-22T23:23:28.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.453+0000 7fd767fff640 1 -- 10.20.193.6:0/2388511178 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd77c003a70 con 0x7fd78c0c99a0 2026-02-22T23:23:28.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.453+0000 7fd767fff640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd75c078e00 0x7fd75c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.453+0000 7fd78b7fe640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd75c078e00 0x7fd75c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.454+0000 7fd78b7fe640 10 monclient: get_auth_request con 0x7fd75c078e00 auth_method 0 2026-02-22T23:23:28.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.454+0000 7fd78b7fe640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd75c078e00 0x7fd75c07b2c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd778039000 tx=0x7fd778033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.454+0000 7fd767fff640 10 client.4485.objecter ms_handle_connect 0x7fd75c078e00 2026-02-22T23:23:28.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.456+0000 7fd767fff640 1 -- 10.20.193.6:0/2388511178 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fd77c05e100 con 0x7fd78c0c99a0 2026-02-22T23:23:28.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.456+0000 7fd767fff640 10 client.4485.objecter ms_dispatch2 0x7fd78c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.456+0000 7fd767fff640 3 client.4485.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.456+0000 7fd767fff640 3 client.4485.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.456+0000 7fd767fff640 20 client.4485.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.509+0000 7fac8bda9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:28.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.509+0000 7fac8bda9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:28.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.509+0000 7fac8bda9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.509+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fac840cc500 con 0x7fac8415d730 2026-02-22T23:23:28.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.515+0000 7f4acbde4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:28.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.515+0000 7f4acbde4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:28.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.515+0000 7f4acbde4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.515+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7f4ac41542b0 con 0x7f4aac004900 2026-02-22T23:23:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.544+0000 7fac727fc640 1 -- 10.20.193.6:0/845472967 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fac80061a10 con 0x7fac8415d730 2026-02-22T23:23:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.544+0000 7fac727fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.544+0000 7fac727fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.544+0000 7f59a97fa640 1 -- 10.20.193.6:0/2394814723 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f59b40cbee0 con 0x7f59b415d2c0 2026-02-22T23:23:28.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.544+0000 7f59a97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.544+0000 7f59a97fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.547 INFO:teuthology.orchestra.run.trial006.stdout:94489280514 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 10 librados: watch_flush enter 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 10 librados: watch_flush exit 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 20 client.4495.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 10 client.4495.objecter successfully canceled tick 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fac54078e00 msgr2=0x7fac5407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fac54078e00 0x7fac5407b2c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fac8406cdf0 tx=0x7fac74031000 comp rx=0 tx=0).stop 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 10 monclient: shutdown 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fac8415d730 msgr2=0x7fac840c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fac8415d730 0x7fac840c79c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac8000b990 tx=0x7fac8000be60 comp rx=0 tx=0).stop 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 shutdown_connections 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fac54078e00 0x7fac5407b2c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fac8415d730 0x7fac840c79c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fac8415d2c0 0x7fac8414ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 --2- 10.20.193.6:0/845472967 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fac841527f0 0x7fac841f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 >> 10.20.193.6:0/845472967 conn(0x7fac840ce730 msgr2=0x7fac840cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 shutdown_connections 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 -- 10.20.193.6:0/845472967 wait complete. 2026-02-22T23:23:28.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.546+0000 7fac8bda9640 1 librados: shutdown 2026-02-22T23:23:28.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.554+0000 7fd767fff640 1 -- 10.20.193.6:0/2388511178 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd77c0627c0 con 0x7fd78c0c99a0 2026-02-22T23:23:28.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.554+0000 7fd767fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.554+0000 7fd767fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280514 for osd.5 2026-02-22T23:23:28.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.558+0000 7f4ab27fc640 1 -- 10.20.193.6:0/3351743640 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f4ab4022020 con 0x7f4aac004900 2026-02-22T23:23:28.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.558+0000 7f4ab27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:28.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.558+0000 7f4ab27fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.561 INFO:teuthology.orchestra.run.trial006.stdout:94489280515 2026-02-22T23:23:28.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 10 librados: watch_flush enter 2026-02-22T23:23:28.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 10 librados: watch_flush exit 2026-02-22T23:23:28.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 20 client.4482.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 10 client.4482.objecter successfully canceled tick 2026-02-22T23:23:28.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a94078e50 msgr2=0x7f4a9407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a94078e50 0x7f4a9407b310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4ab803b000 tx=0x7f4ab8035000 comp rx=0 tx=0).stop 2026-02-22T23:23:28.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 10 monclient: shutdown 2026-02-22T23:23:28.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac004900 msgr2=0x7f4ac4151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac004900 0x7f4ac4151560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400c770 tx=0x7f4ab400cc40 comp rx=0 tx=0).stop 2026-02-22T23:23:28.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 shutdown_connections 2026-02-22T23:23:28.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4a94078e50 0x7f4a9407b310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4ac41f2180 0x7f4ac40c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4ac4151aa0 0x7f4ac40c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 --2- 10.20.193.6:0/3351743640 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4aac004900 0x7f4ac4151560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 >> 10.20.193.6:0/3351743640 conn(0x7f4ac40cc020 msgr2=0x7f4ac40713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 shutdown_connections 2026-02-22T23:23:28.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 -- 10.20.193.6:0/3351743640 wait complete. 2026-02-22T23:23:28.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.560+0000 7f4acbde4640 1 librados: shutdown 2026-02-22T23:23:28.571 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280515 for osd.12 2026-02-22T23:23:28.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.608+0000 7fad167fc640 1 -- 10.20.193.6:0/1574591571 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fad1c082e20 con 0x7fad281527f0 2026-02-22T23:23:28.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.608+0000 7fad167fc640 10 client.4550.objecter ms_dispatch2 0x7fad28000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.608+0000 7fad167fc640 3 client.4550.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.608+0000 7fad167fc640 3 client.4550.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.608+0000 7fad167fc640 20 client.4550.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.615+0000 7fad167fc640 1 -- 10.20.193.6:0/1574591571 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fad280739a0 con 0x7fad281527f0 2026-02-22T23:23:28.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.615+0000 7fad167fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.615+0000 7fad167fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.624+0000 7ff094ff9640 1 -- 10.20.193.6:0/2089095073 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff084019460 con 0x7ff098074d60 2026-02-22T23:23:28.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.624+0000 7ff094ff9640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff068078e50 0x7ff06807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.625+0000 7ff0977fe640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff068078e50 0x7ff06807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.625+0000 7ff0977fe640 10 monclient: get_auth_request con 0x7ff068078e50 auth_method 0 2026-02-22T23:23:28.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.625+0000 7ff094ff9640 10 client.4541.objecter ms_handle_connect 0x7ff068078e50 2026-02-22T23:23:28.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.625+0000 7ff0977fe640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff068078e50 0x7ff06807b310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff09815d870 tx=0x7ff088006ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.627+0000 7ff094ff9640 1 -- 10.20.193.6:0/2089095073 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7ff08405de40 con 0x7ff098074d60 2026-02-22T23:23:28.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.627+0000 7ff094ff9640 10 client.4541.objecter ms_dispatch2 0x7ff098000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.627+0000 7ff094ff9640 3 client.4541.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.627+0000 7ff094ff9640 3 client.4541.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.627+0000 7ff094ff9640 20 client.4541.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.630+0000 7f59bac1a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:28.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.630+0000 7f59bac1a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:28.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.630+0000 7f59bac1a640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:28.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.630+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f59b4152bd0 con 0x7f59b415d2c0 2026-02-22T23:23:28.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.632+0000 7f59a97fa640 1 -- 10.20.193.6:0/2394814723 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f599c086dc0 con 0x7f59b415d2c0 2026-02-22T23:23:28.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.632+0000 7f59a97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:28.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.632+0000 7f59a97fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.634 INFO:teuthology.orchestra.run.trial006.stdout:94489280514 2026-02-22T23:23:28.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 10 librados: watch_flush enter 2026-02-22T23:23:28.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 10 librados: watch_flush exit 2026-02-22T23:23:28.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 20 client.4501.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 10 client.4501.objecter successfully canceled tick 2026-02-22T23:23:28.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5984078e50 msgr2=0x7f598407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5984078e50 0x7f598407b310 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f59a40098a0 tx=0x7f59a4006d90 comp rx=0 tx=0).stop 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 10 monclient: shutdown 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59b415d2c0 msgr2=0x7f59b41f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59b415d2c0 0x7f59b41f5190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f599c006fd0 tx=0x7f599c0047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 shutdown_connections 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5984078e50 0x7f598407b310 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59b415d730 0x7f59b41f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59b415d2c0 0x7f59b41f5190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 --2- 10.20.193.6:0/2394814723 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59b41527f0 0x7f59b414dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 >> 10.20.193.6:0/2394814723 conn(0x7f59b40ce730 msgr2=0x7f59b415dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 shutdown_connections 2026-02-22T23:23:28.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 -- 10.20.193.6:0/2394814723 wait complete. 2026-02-22T23:23:28.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.634+0000 7f59bac1a640 1 librados: shutdown 2026-02-22T23:23:28.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.635+0000 7ff094ff9640 1 -- 10.20.193.6:0/2089095073 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff084062500 con 0x7ff098074d60 2026-02-22T23:23:28.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.635+0000 7ff094ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.635+0000 7ff094ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.639+0000 7fd7921d3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:28.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.639+0000 7fd7921d3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:28.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.639+0000 7fd7921d3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:28.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.639+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd78c1583f0 con 0x7fd78c0c99a0 2026-02-22T23:23:28.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.642+0000 7fd767fff640 1 -- 10.20.193.6:0/2388511178 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd77c0620c0 con 0x7fd78c0c99a0 2026-02-22T23:23:28.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.642+0000 7fd767fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:28.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.642+0000 7fd767fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.644 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:28.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.642+0000 7f7d66ffd640 1 -- 10.20.193.6:0/1949183421 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7d7003f050 con 0x7f7d801527f0 2026-02-22T23:23:28.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280514 for osd.3 2026-02-22T23:23:28.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.643+0000 7f7d66ffd640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d50078e50 0x7f7d5007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:28.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.643+0000 7f7d85a1d640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d50078e50 0x7f7d5007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:28.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.643+0000 7f7d85a1d640 10 monclient: get_auth_request con 0x7f7d50078e50 auth_method 0 2026-02-22T23:23:28.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.643+0000 7f7d85a1d640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d50078e50 0x7f7d5007b310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d74006fd0 tx=0x7f7d74006d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:28.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.643+0000 7f7d66ffd640 10 client.4547.objecter ms_handle_connect 0x7f7d50078e50 2026-02-22T23:23:28.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 10 librados: watch_flush enter 2026-02-22T23:23:28.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 10 librados: watch_flush exit 2026-02-22T23:23:28.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 20 client.4485.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 10 client.4485.objecter successfully canceled tick 2026-02-22T23:23:28.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd75c078e00 msgr2=0x7fd75c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd75c078e00 0x7fd75c07b2c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd778039000 tx=0x7fd778033000 comp rx=0 tx=0).stop 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 10 monclient: shutdown 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd78c0c99a0 msgr2=0x7fd78c154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd78c0c99a0 0x7fd78c154f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00c700 tx=0x7fd77c00cbd0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 shutdown_connections 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd75c078e00 0x7fd75c07b2c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd78c15a5e0 0x7fd78c155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd78c0c99a0 0x7fd78c154f20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 --2- 10.20.193.6:0/2388511178 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd78c06ff20 0x7fd78c14bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 >> 10.20.193.6:0/2388511178 conn(0x7fd78c0cd780 msgr2=0x7fd78c14ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 shutdown_connections 2026-02-22T23:23:28.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 -- 10.20.193.6:0/2388511178 wait complete. 2026-02-22T23:23:28.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.644+0000 7fd7921d3640 1 librados: shutdown 2026-02-22T23:23:28.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7f7d66ffd640 1 -- 10.20.193.6:0/1949183421 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f7d70082f10 con 0x7f7d801527f0 2026-02-22T23:23:28.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7f7d66ffd640 10 client.4547.objecter ms_dispatch2 0x7f7d80000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:28.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7f7d66ffd640 3 client.4547.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:28.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7f7d66ffd640 3 client.4547.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:28.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7f7d66ffd640 20 client.4547.objecter dump_active .. 0 homeless 2026-02-22T23:23:28.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7fb25b7fe640 1 -- 10.20.193.6:0/88144848 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fb26c061f70 con 0x7fb27c15a5e0 2026-02-22T23:23:28.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7fb25b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:28.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.645+0000 7fb25b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.653 INFO:teuthology.orchestra.run.trial006.stdout:94489280515 2026-02-22T23:23:28.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 10 librados: watch_flush enter 2026-02-22T23:23:28.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 10 librados: watch_flush exit 2026-02-22T23:23:28.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 20 client.4538.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 10 client.4538.objecter successfully canceled tick 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb248078e50 msgr2=0x7fb24807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb248078e50 0x7fb24807b310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb270039000 tx=0x7fb270033000 comp rx=0 tx=0).stop 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 10 monclient: shutdown 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb27c15a5e0 msgr2=0x7fb27c0bea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb27c15a5e0 0x7fb27c0bea20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00b970 tx=0x7fb26c00be40 comp rx=0 tx=0).stop 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 shutdown_connections 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb248078e50 0x7fb24807b310 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb27c15a5e0 0x7fb27c0bea20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb27c0c99a0 0x7fb27c1f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 --2- 10.20.193.6:0/88144848 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb27c06ff20 0x7fb27c0c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 >> 10.20.193.6:0/88144848 conn(0x7fb27c0cd780 msgr2=0x7fb27c14e520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 shutdown_connections 2026-02-22T23:23:28.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 -- 10.20.193.6:0/88144848 wait complete. 2026-02-22T23:23:28.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.648+0000 7fb28102b640 1 librados: shutdown 2026-02-22T23:23:28.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.653+0000 7f7d66ffd640 1 -- 10.20.193.6:0/1949183421 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7d700875d0 con 0x7f7d801527f0 2026-02-22T23:23:28.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.653+0000 7f7d66ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:28.657 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.653+0000 7f7d66ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:28.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.6 2026-02-22T23:23:28.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280515 for osd.15 2026-02-22T23:23:28.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.702+0000 7fad2fc58640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:28.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.702+0000 7fad2fc58640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:28.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.702+0000 7fad2fc58640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.702+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fad2806bde0 con 0x7fad281527f0 2026-02-22T23:23:28.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.708+0000 7fad167fc640 1 -- 10.20.193.6:0/1574591571 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fad2815fd90 con 0x7fad281527f0 2026-02-22T23:23:28.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.708+0000 7fad167fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:28.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.708+0000 7fad167fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.710 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:28.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 10 librados: watch_flush enter 2026-02-22T23:23:28.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 10 librados: watch_flush exit 2026-02-22T23:23:28.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 20 client.4550.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 10 client.4550.objecter successfully canceled tick 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf8078e50 msgr2=0x7facf807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf8078e50 0x7facf807b310 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad18006fd0 tx=0x7fad18006d40 comp rx=0 tx=0).stop 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 10 monclient: shutdown 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad281527f0 msgr2=0x7fad281e81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad281527f0 0x7fad281e81d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009970 tx=0x7fad1c0047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 shutdown_connections 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7facf8078e50 0x7facf807b310 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fad2815f900 0x7fad281f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fad281527f0 0x7fad281e81d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 --2- 10.20.193.6:0/1574591571 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fad28073530 0x7fad281f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 >> 10.20.193.6:0/1574591571 conn(0x7fad280ce730 msgr2=0x7fad280cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 shutdown_connections 2026-02-22T23:23:28.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 -- 10.20.193.6:0/1574591571 wait complete. 2026-02-22T23:23:28.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.710+0000 7fad2fc58640 1 librados: shutdown 2026-02-22T23:23:28.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.11 2026-02-22T23:23:28.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.722+0000 7ff09e340640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:28.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.722+0000 7ff09e340640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:28.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.722+0000 7ff09e340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.722+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff098159890 con 0x7ff098074d60 2026-02-22T23:23:28.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.724+0000 7ff094ff9640 1 -- 10.20.193.6:0/2089095073 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7ff084061e00 con 0x7ff098074d60 2026-02-22T23:23:28.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.724+0000 7ff094ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:28.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.724+0000 7ff094ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.726 INFO:teuthology.orchestra.run.trial006.stdout:90194313218 2026-02-22T23:23:28.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 10 librados: watch_flush enter 2026-02-22T23:23:28.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 10 librados: watch_flush exit 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 20 client.4541.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 10 client.4541.objecter successfully canceled tick 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff068078e50 msgr2=0x7ff06807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff068078e50 0x7ff06807b310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff09815d870 tx=0x7ff088006ca0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 10 monclient: shutdown 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff098074d60 msgr2=0x7ff098142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff098074d60 0x7ff098142f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b500 tx=0x7ff08400b9d0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 shutdown_connections 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff068078e50 0x7ff06807b310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff098156250 0x7ff09814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff098155e70 0x7ff098144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 --2- 10.20.193.6:0/2089095073 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff098074d60 0x7ff098142f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 >> 10.20.193.6:0/2089095073 conn(0x7ff0980ce700 msgr2=0x7ff09815f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 shutdown_connections 2026-02-22T23:23:28.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 -- 10.20.193.6:0/2089095073 wait complete. 2026-02-22T23:23:28.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.726+0000 7ff09e340640 1 librados: shutdown 2026-02-22T23:23:28.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.10 2026-02-22T23:23:28.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.739+0000 7f7d884a9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-22T23:23:28.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.739+0000 7f7d884a9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-22T23:23:28.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.739+0000 7f7d884a9640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:28.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.739+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7d800cc500 con 0x7f7d801527f0 2026-02-22T23:23:28.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.742+0000 7f7d66ffd640 1 -- 10.20.193.6:0/1949183421 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7d70086ed0 con 0x7f7d801527f0 2026-02-22T23:23:28.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.742+0000 7f7d66ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-22T23:23:28.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.742+0000 7f7d66ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:28.744 INFO:teuthology.orchestra.run.trial006.stdout:90194313218 2026-02-22T23:23:28.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 10 librados: watch_flush enter 2026-02-22T23:23:28.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 10 librados: watch_flush exit 2026-02-22T23:23:28.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 20 client.4547.objecter shutdown clearing up homeless session... 2026-02-22T23:23:28.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 10 client.4547.objecter successfully canceled tick 2026-02-22T23:23:28.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d50078e50 msgr2=0x7f7d5007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d50078e50 0x7f7d5007b310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d74006fd0 tx=0x7f7d74006d40 comp rx=0 tx=0).stop 2026-02-22T23:23:28.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 10 monclient: shutdown 2026-02-22T23:23:28.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:28.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d801527f0 msgr2=0x7f7d801f41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:28.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d801527f0 0x7f7d801f41c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009aa0 tx=0x7f7d700047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 shutdown_connections 2026-02-22T23:23:28.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d50078e50 0x7f7d5007b310 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d8015d730 0x7f7d800c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d8015d2c0 0x7f7d8014ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 --2- 10.20.193.6:0/1949183421 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d801527f0 0x7f7d801f41c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:28.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 >> 10.20.193.6:0/1949183421 conn(0x7f7d800ce730 msgr2=0x7f7d800cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:28.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 shutdown_connections 2026-02-22T23:23:28.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 -- 10.20.193.6:0/1949183421 wait complete. 2026-02-22T23:23:28.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:28.744+0000 7f7d884a9640 1 librados: shutdown 2026-02-22T23:23:28.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.2 2026-02-22T23:23:29.168 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-22T23:23:29.206 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.238+0000 7f695263c640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.238+0000 7f695263c640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.238+0000 7f695263c640 10 monclient: monmap: 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.239288+0000 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.239288+0000 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.240 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.241 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.241 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.241 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.241 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.241 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.241 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 1 Processor -- start 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 1 -- start start 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient: init 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient: _add_conns ranks=[1,2,0,3,5,4] 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f694c1527f0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f694c073530 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f694c15fad0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694c0739a0 con 0x7f694c15fad0 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694c15d9d0 con 0x7f694c1527f0 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694c073c20 con 0x7f694c073530 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.239+0000 7f695263c640 10 monclient(hunting): authenticate will time out at 1083.966724s 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.240+0000 7f694b7fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f694c073530 0x7f694c15f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36210/0 (socket says 10.20.193.6:36210) 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.240+0000 7f694b7fe640 1 -- 10.20.193.6:0/477714418 learned_addr learned my addr 10.20.193.6:0/477714418 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.240+0000 7f694bfff640 1 --1- 10.20.193.6:0/477714418 >> v1:10.20.193.40:6789/0 conn(0x7f694c1527f0 0x7f694c15d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60300/0 (socket says 10.20.193.6:60300) 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513080157 0 0) 0x7f694c15d9d0 con 0x7f694c1527f0 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): my global_id is 4504 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6930003620 con 0x7f694c1527f0 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139787075 0 0) 0x7f694c073c20 con 0x7f694c073530 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 10 monclient(hunting): my global_id is 4553 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.241+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f694c15d9d0 con 0x7f694c073530 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.242+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593345002 0 0) 0x7f6940002ca0 con 0x7f694c15fad0 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.242+0000 7f694affd640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.242+0000 7f694affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.242+0000 7f694affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.242+0000 7f694affd640 10 monclient(hunting): my global_id is 4488 2026-02-22T23:23:29.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.242+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f694c073c20 con 0x7f694c15fad0 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3633985783 0 0) 0x7f6930003620 con 0x7f694c1527f0 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 >> v1:10.20.193.80:6789/0 conn(0x7f694c073530 legacy=0x7f694c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 >> v1:10.20.193.6:6789/0 conn(0x7f694c15fad0 legacy=0x7f694c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:23:29.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c073f00 con 0x7f694c1527f0 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f6938002f00 con 0x7f694c1527f0 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f695263c640 5 monclient: authenticate success, global_id 4504 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f695263c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: dump: 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.255 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 >> v1:10.20.193.40:6789/0 conn(0x7f694c1527f0 legacy=0x7f694c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:29.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6930003fe0 0x7f6930004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): picked mon.b con 0x7f6930003fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930004e40 0x7f69300052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): picked mon.a con 0x7f6930004e40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6930009ab0 0x7f693000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): picked mon.c con 0x7f6930009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f693000c570 con 0x7f6930004e40 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f693000c6f0 con 0x7f6930003fe0 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f693000c8f0 con 0x7f6930009ab0 2026-02-22T23:23:29.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f694affd640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.243+0000 7f695263c640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.244+0000 7f6950bb2640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6930009ab0 0x7f693000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.244+0000 7f6950bb2640 10 monclient(hunting): get_auth_request con 0x7f6930009ab0 auth_method 0 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.244+0000 7f6950bb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.244+0000 7f6950bb2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.244+0000 7f6950bb2640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.244+0000 7f694bfff640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6930003fe0 0x7f6930004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930004e40 0x7f69300052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694bfff640 10 monclient(hunting): get_auth_request con 0x7f6930003fe0 auth_method 0 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 10 monclient(hunting): get_auth_request con 0x7f6930004e40 auth_method 0 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f694b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient(hunting): handle_auth_done global_id 4504 payload 293 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 1 -- 10.20.193.6:0/477714418 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6930003fe0 msgr2=0x7f6930004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6930003fe0 0x7f6930004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 1 -- 10.20.193.6:0/477714418 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930004e40 msgr2=0x7f69300052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930004e40 0x7f69300052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 1 monclient: found mon.c 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 1 -- 10.20.193.6:0/477714418 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6930003620 con 0x7f6930009ab0 2026-02-22T23:23:29.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.245+0000 7f6950bb2640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.246+0000 7f6950bb2640 1 -- 10.20.193.6:0/477714418 reap_dead start 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.246+0000 7f6950bb2640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6930009ab0 0x7f693000bea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f694000c490 tx=0x7f694002f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1caf166a6315595c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694000c0e0 con 0x7f6930009ab0 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 10 monclient: dump: 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.261 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6940003c70 con 0x7f6930009ab0 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f695263c640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 1 -- 10.20.193.6:0/477714418 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940005310 con 0x7f6930009ab0 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f695263c640 4 set_mon_vals no callback set 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f694affd640 10 monclient: dump: 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f695263c640 10 monclient: shutdown 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f695263c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f695263c640 1 -- 10.20.193.6:0/477714418 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6930009ab0 msgr2=0x7f693000bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.248+0000 7f695263c640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6930009ab0 0x7f693000bea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f694000c490 tx=0x7f694002f7c0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 -- 10.20.193.6:0/477714418 shutdown_connections 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 --2- 10.20.193.6:0/477714418 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6930009ab0 0x7f693000bea0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 -- 10.20.193.6:0/477714418 >> 10.20.193.6:0/477714418 conn(0x7f694c0ce730 msgr2=0x7f694c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 -- 10.20.193.6:0/477714418 shutdown_connections 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 -- 10.20.193.6:0/477714418 wait complete. 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 10 monclient: monmap: 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.264 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 librados: starting msgr at 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 librados: starting objecter 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 Processor -- start 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 -- start start 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 librados: setting wanted keys 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 1 librados: calling monclient init 2026-02-22T23:23:29.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.249+0000 7f695263c640 10 monclient: init 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930009ab0 0x7f694c14e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): picked mon.noname-a con 0x7f6930009ab0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f694c14eec0 0x7f694c1f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): picked mon.noname-b con 0x7f694c14eec0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f694c1f44d0 0x7f694c1f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): picked mon.noname-c con 0x7f694c1f44d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0744c0 con 0x7f6930009ab0 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f694c074b10 con 0x7f694c14eec0 2026-02-22T23:23:29.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f694c0747a0 con 0x7f694c1f44d0 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f695263c640 10 monclient(hunting): authenticate will time out at 1083.977558s 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f694bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930009ab0 0x7f694c14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f694b7fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f694c14eec0 0x7f694c1f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.250+0000 7f6950bb2640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f694c1f44d0 0x7f694c1f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930009ab0 0x7f694c14e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39816/0 (socket says 10.20.193.6:39816) 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 1 -- 10.20.193.6:0/2215204681 learned_addr learned my addr 10.20.193.6:0/2215204681 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 10 monclient(hunting): get_auth_request con 0x7f6930009ab0 auth_method 0 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694b7fe640 10 monclient(hunting): get_auth_request con 0x7f694c14eec0 auth_method 0 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f6950bb2640 10 monclient(hunting): get_auth_request con 0x7f694c1f44d0 auth_method 0 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f6950bb2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f6950bb2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f6950bb2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f6950bb2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f6950bb2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f6950bb2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.251+0000 7f694b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 10 monclient(hunting): handle_auth_done global_id 4507 payload 824 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 1 -- 10.20.193.6:0/2215204681 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f694c1f44d0 msgr2=0x7f694c1f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f694c1f44d0 0x7f694c1f2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 1 -- 10.20.193.6:0/2215204681 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930009ab0 msgr2=0x7f694c14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930009ab0 0x7f694c14e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.252+0000 7f694b7fe640 1 -- 10.20.193.6:0/2215204681 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c1f19b0 con 0x7f694c14eec0 2026-02-22T23:23:29.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.253+0000 7f694b7fe640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f694c14eec0 0x7f694c1f2a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f693c004f30 tx=0x7f693c007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.253+0000 7f6948ff9640 10 client.?.objecter ms_handle_connect 0x7f694c14eec0 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.253+0000 7f6948ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 1 -- 10.20.193.6:0/2215204681 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c01a070 con 0x7f694c14eec0 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: dump: 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.273 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 1 -- 10.20.193.6:0/2215204681 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693c00f690 con 0x7f694c14eec0 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 1 -- 10.20.193.6:0/2215204681 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c00fa40 con 0x7f694c14eec0 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f694a7fc640 4 set_mon_vals no callback set 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f6948ff9640 10 monclient: dump: 2026-02-22T23:23:29.274 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 5 monclient: authenticate success, global_id 4507 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 10 monclient: _renew_subs 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c1f1c90 con 0x7f694c14eec0 2026-02-22T23:23:29.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 10 client.4507.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 10 monclient: _renew_subs 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c1f2250 con 0x7f694c14eec0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.255+0000 7f695263c640 1 librados: init done 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.256+0000 7f695263c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.256+0000 7f695263c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.256+0000 7f695263c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.256+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694c074fa0 con 0x7f694c14eec0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.267+0000 7f6948ff9640 1 -- 10.20.193.6:0/2215204681 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f693c00fbe0 con 0x7f694c14eec0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.267+0000 7f6948ff9640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f691c078e30 0x7f691c07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.268+0000 7f694bfff640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f691c078e30 0x7f691c07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.268+0000 7f694bfff640 10 monclient: get_auth_request con 0x7f691c078e30 auth_method 0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.268+0000 7f6948ff9640 10 client.4507.objecter ms_handle_connect 0x7f691c078e30 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.268+0000 7f694bfff640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f691c078e30 0x7f691c07b2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6938007cc0 tx=0x7f6938002870 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.269+0000 7f6948ff9640 1 -- 10.20.193.6:0/2215204681 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f693c05d380 con 0x7f694c14eec0 2026-02-22T23:23:29.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.269+0000 7f6948ff9640 10 client.4507.objecter ms_dispatch2 0x7f694c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.269+0000 7f6948ff9640 3 client.4507.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.269+0000 7f6948ff9640 3 client.4507.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.269+0000 7f6948ff9640 20 client.4507.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.278+0000 7f96b7534640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.278+0000 7f96b7534640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.278+0000 7f96b7534640 10 monclient: monmap: 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.279369+0000 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.279369+0000 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.280 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.281 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.281 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.281 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 1 Processor -- start 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 1 -- start start 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient: init 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient: _add_conns ranks=[5,4,0,2,3,1] 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f96b006ff20 0x7f96b0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient(hunting): picked mon.noname-c con 0x7f96b006ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f96b00c99a0 0x7f96b0159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient(hunting): picked mon.noname-b con 0x7f96b00c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f96b015a410 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:29.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b015a860 con 0x7f96b015a410 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f96b0159a40 con 0x7f96b00c99a0 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f96b00c7e90 con 0x7f96b006ff20 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.279+0000 7f96b7534640 10 monclient(hunting): authenticate will time out at 1084.006804s 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f96b006ff20 0x7f96b0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b5aaa640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f96b015a410 0x7f96b014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35518/0 (socket says 10.20.193.6:35518) 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b5aaa640 1 -- 10.20.193.6:0/3538825745 learned_addr learned my addr 10.20.193.6:0/3538825745 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 10 monclient(hunting): get_auth_request con 0x7f96b006ff20 auth_method 0 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.280+0000 7f96b52a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f969ffff640 1 -- 10.20.193.6:0/3538825745 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206074089 0 0) 0x7f96b015a860 con 0x7f96b015a410 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 10 monclient(hunting): handle_auth_done global_id 4559 payload 274 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 1 -- 10.20.193.6:0/3538825745 >> v2:10.20.193.40:3300/0 conn(0x7f96b00c99a0 msgr2=0x7f96b0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 1 --2- 10.20.193.6:0/3538825745 >> v2:10.20.193.40:3300/0 conn(0x7f96b00c99a0 0x7f96b0159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 1 -- 10.20.193.6:0/3538825745 >> v1:10.20.193.6:6789/0 conn(0x7f96b015a410 legacy=0x7f96b014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f96b52a9640 1 -- 10.20.193.6:0/3538825745 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b00c8170 con 0x7f96b006ff20 2026-02-22T23:23:29.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.281+0000 7f969ffff640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.282+0000 7f96b52a9640 1 --2- 10.20.193.6:0/3538825745 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b0070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f96a00098a0 tx=0x7f96a002ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f58b023f2044a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 1 -- 10.20.193.6:0/3538825745 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a003c070 con 0x7f96b006ff20 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: dump: 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.289 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 1 -- 10.20.193.6:0/3538825745 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a002fae0 con 0x7f96b006ff20 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 5 monclient: authenticate success, global_id 4559 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 1 -- 10.20.193.6:0/3538825745 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0037440 con 0x7f96b006ff20 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 4 set_mon_vals no callback set 2026-02-22T23:23:29.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f969ffff640 10 monclient: dump: 2026-02-22T23:23:29.292 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.292 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.292 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.292 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 10 monclient: shutdown 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 -- 10.20.193.6:0/3538825745 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 msgr2=0x7f96b0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 --2- 10.20.193.6:0/3538825745 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b0070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f96a00098a0 tx=0x7f96a002ee10 comp rx=0 tx=0).stop 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 -- 10.20.193.6:0/3538825745 shutdown_connections 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 --2- 10.20.193.6:0/3538825745 >> v2:10.20.193.40:3300/0 conn(0x7f96b00c99a0 0x7f96b0159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 --2- 10.20.193.6:0/3538825745 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b0070300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 -- 10.20.193.6:0/3538825745 >> 10.20.193.6:0/3538825745 conn(0x7f96b00cd780 msgr2=0x7f96b00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 -- 10.20.193.6:0/3538825745 shutdown_connections 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 1 -- 10.20.193.6:0/3538825745 wait complete. 2026-02-22T23:23:29.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.284+0000 7f96b7534640 10 monclient: monmap: 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.295 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 librados: starting msgr at 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 librados: starting objecter 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 Processor -- start 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 -- start start 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 librados: setting wanted keys 2026-02-22T23:23:29.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 librados: calling monclient init 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient: init 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b014cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): picked mon.noname-c con 0x7f96b006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f96b00c99a0 0x7f96b0152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): picked mon.noname-b con 0x7f96b00c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f96b015a410 0x7f96b0153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): picked mon.noname-a con 0x7f96b015a410 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f96b00c8a10 con 0x7f96b015a410 2026-02-22T23:23:29.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f96b00c8730 con 0x7f96b00c99a0 2026-02-22T23:23:29.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0150520 con 0x7f96b006ff20 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b7534640 10 monclient(hunting): authenticate will time out at 1084.012672s 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b5aaa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f96b015a410 0x7f96b0153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.285+0000 7f96b52a9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b014cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f96b015a410 0x7f96b0153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39818/0 (socket says 10.20.193.6:39818) 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 1 -- 10.20.193.6:0/2095917272 learned_addr learned my addr 10.20.193.6:0/2095917272 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 10 monclient(hunting): get_auth_request con 0x7f96b015a410 auth_method 0 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b52a9640 10 monclient(hunting): get_auth_request con 0x7f96b006ff20 auth_method 0 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b52a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b52a9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b52a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b5aaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b52a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b52a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.286+0000 7f96b52a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 10 monclient(hunting): handle_auth_done global_id 4562 payload 824 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 1 -- 10.20.193.6:0/2095917272 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f96b00c99a0 msgr2=0x7f96b0152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f96b00c99a0 0x7f96b0152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 1 -- 10.20.193.6:0/2095917272 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f96b015a410 msgr2=0x7f96b0153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f96b015a410 0x7f96b0153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.287+0000 7f96b52a9640 1 -- 10.20.193.6:0/2095917272 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b00beb50 con 0x7f96b006ff20 2026-02-22T23:23:29.302 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:23:29.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.288+0000 7f96b52a9640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b014cd70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96a0036000 tx=0x7f96a00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.288+0000 7f969dffb640 10 client.?.objecter ms_handle_connect 0x7f96b006ff20 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.288+0000 7f969dffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f969dffb640 1 -- 10.20.193.6:0/2095917272 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0046020 con 0x7f96b006ff20 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f969dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f969dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f969dffb640 10 monclient: dump: 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.307 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f969dffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f969dffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f969dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f96b7534640 5 monclient: authenticate success, global_id 4562 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f96b7534640 10 monclient: _renew_subs 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f96b7534640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.289+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b00bee30 con 0x7f96b006ff20 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 10 client.4562.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 10 monclient: _renew_subs 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b00bf3f0 con 0x7f96b006ff20 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 1 librados: init done 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f969dffb640 1 -- 10.20.193.6:0/2095917272 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a002fae0 con 0x7f96b006ff20 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f969dffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f969dffb640 1 -- 10.20.193.6:0/2095917272 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a003c040 con 0x7f96b006ff20 2026-02-22T23:23:29.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f969dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f969dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f969dffb640 10 monclient: dump: 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.310 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f969f7fe640 4 set_mon_vals no callback set 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.290+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b0070300 con 0x7f96b006ff20 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.301+0000 7f969dffb640 1 -- 10.20.193.6:0/2095917272 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f96a003f030 con 0x7f96b006ff20 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.302+0000 7f969dffb640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9680078e30 0x7f968007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.302+0000 7f96b4aa8640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9680078e30 0x7f968007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.302+0000 7f96b4aa8640 10 monclient: get_auth_request con 0x7f9680078e30 auth_method 0 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.302+0000 7f96b4aa8640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9680078e30 0x7f968007b2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96b00c5ce0 tx=0x7f96a4006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.302+0000 7f969dffb640 10 client.4562.objecter ms_handle_connect 0x7f9680078e30 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.303+0000 7f969dffb640 1 -- 10.20.193.6:0/2095917272 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f96a00833b0 con 0x7f96b006ff20 2026-02-22T23:23:29.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.303+0000 7f969dffb640 10 client.4562.objecter ms_dispatch2 0x7f96b0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.303+0000 7f969dffb640 3 client.4562.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.303+0000 7f969dffb640 3 client.4562.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.303+0000 7f969dffb640 20 client.4562.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.334 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-22T23:23:29.334 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-22T23:23:29.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.369+0000 7f6948ff9640 1 -- 10.20.193.6:0/2215204681 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f693c023080 con 0x7f694c14eec0 2026-02-22T23:23:29.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.369+0000 7f6948ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.369+0000 7f6948ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.371+0000 7f279e2ea640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.371+0000 7f279e2ea640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.371+0000 7f279e2ea640 10 monclient: monmap: 2026-02-22T23:23:29.372 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.372 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.372389+0000 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.372389+0000 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.372+0000 7f279e2ea640 1 Processor -- start 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.372+0000 7f279e2ea640 1 -- start start 2026-02-22T23:23:29.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.372+0000 7f279e2ea640 10 monclient: init 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient: _add_conns ranks=[2,5,4,1,3,0] 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f27981527f0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f2798073530 0x7f279815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient(hunting): picked mon.noname-c con 0x7f2798073530 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f279815f900 0x7f27980ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient(hunting): picked mon.noname-b con 0x7f279815f900 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27980739a0 con 0x7f27981527f0 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2798074b10 con 0x7f279815f900 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f279815fd90 con 0x7f2798073530 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279e2ea640 10 monclient(hunting): authenticate will time out at 1084.100114s 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279c860640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f279815f900 0x7f27980ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279c860640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f279815f900 0x7f27980ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36514/0 (socket says 10.20.193.6:36514) 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279c860640 1 -- 10.20.193.6:0/2274840368 learned_addr learned my addr 10.20.193.6:0/2274840368 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279c860640 10 monclient(hunting): get_auth_request con 0x7f279815f900 auth_method 0 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279c860640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279c860640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.373+0000 7f279c860640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 10 monclient(hunting): handle_auth_done global_id 4510 payload 274 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 1 -- 10.20.193.6:0/2274840368 >> v2:10.20.193.80:3300/0 conn(0x7f2798073530 msgr2=0x7f279815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 1 --2- 10.20.193.6:0/2274840368 >> v2:10.20.193.80:3300/0 conn(0x7f2798073530 0x7f279815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 1 -- 10.20.193.6:0/2274840368 >> v1:10.20.193.80:6789/0 conn(0x7f27981527f0 legacy=0x7f279815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.374+0000 7f279c860640 1 -- 10.20.193.6:0/2274840368 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279815d9d0 con 0x7f279815f900 2026-02-22T23:23:29.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.375+0000 7f279c860640 1 --2- 10.20.193.6:0/2274840368 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f279815f900 0x7f27980ccf70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f279808f510 tx=0x7f278c02f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=421727a4f3fbed70 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 1 -- 10.20.193.6:0/2274840368 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c02fcd0 con 0x7f279815f900 2026-02-22T23:23:29.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: dump: 2026-02-22T23:23:29.378 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 1 -- 10.20.193.6:0/2274840368 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c0373f0 con 0x7f279815f900 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 5 monclient: authenticate success, global_id 4510 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 1 -- 10.20.193.6:0/2274840368 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0377c0 con 0x7f279815f900 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 4 set_mon_vals no callback set 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f2796ffd640 10 monclient: dump: 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 10 monclient: shutdown 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 -- 10.20.193.6:0/2274840368 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f279815f900 msgr2=0x7f27980ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2274840368 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f279815f900 0x7f27980ccf70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f279808f510 tx=0x7f278c02f270 comp rx=0 tx=0).stop 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 -- 10.20.193.6:0/2274840368 shutdown_connections 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2274840368 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f279815f900 0x7f27980ccf70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2274840368 >> v2:10.20.193.80:3300/0 conn(0x7f2798073530 0x7f279815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 -- 10.20.193.6:0/2274840368 >> 10.20.193.6:0/2274840368 conn(0x7f27980ce730 msgr2=0x7f27980ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 -- 10.20.193.6:0/2274840368 shutdown_connections 2026-02-22T23:23:29.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.377+0000 7f279e2ea640 1 -- 10.20.193.6:0/2274840368 wait complete. 2026-02-22T23:23:29.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient: monmap: 2026-02-22T23:23:29.380 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.380 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 librados: starting msgr at 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 librados: starting objecter 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 Processor -- start 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 -- start start 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 librados: setting wanted keys 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 librados: calling monclient init 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient: init 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2798073530 0x7f27981f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): picked mon.noname-a con 0x7f2798073530 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f27981527f0 0x7f27980cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): picked mon.noname-b con 0x7f27981527f0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f279815f900 0x7f27980c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): picked mon.noname-c con 0x7f279815f900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f27980741e0 con 0x7f2798073530 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2798073f00 con 0x7f27981527f0 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f27980744c0 con 0x7f279815f900 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.378+0000 7f279e2ea640 10 monclient(hunting): authenticate will time out at 1084.105819s 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f27981527f0 0x7f27980cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f27981527f0 0x7f27980cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36522/0 (socket says 10.20.193.6:36522) 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 1 -- 10.20.193.6:0/2471548596 learned_addr learned my addr 10.20.193.6:0/2471548596 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 10 monclient(hunting): get_auth_request con 0x7f27981527f0 auth_method 0 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2798073530 0x7f27981f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 10 monclient(hunting): get_auth_request con 0x7f2798073530 auth_method 0 2026-02-22T23:23:29.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f27977fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.379+0000 7f2797fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 10 monclient(hunting): handle_auth_done global_id 4513 payload 824 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 1 -- 10.20.193.6:0/2471548596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f279815f900 msgr2=0x7f27980c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f279815f900 0x7f27980c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 1 -- 10.20.193.6:0/2471548596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2798073530 msgr2=0x7f27981f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2798073530 0x7f27981f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.380+0000 7f27977fe640 1 -- 10.20.193.6:0/2471548596 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981495d0 con 0x7f27981527f0 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.381+0000 7f27977fe640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f27981527f0 0x7f27980cd170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f278800b500 tx=0x7f278800b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.381+0000 7f2794ff9640 10 client.?.objecter ms_handle_connect 0x7f27981527f0 2026-02-22T23:23:29.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.381+0000 7f2794ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 1 -- 10.20.193.6:0/2471548596 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788004480 con 0x7f27981527f0 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: dump: 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 1 -- 10.20.193.6:0/2471548596 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2788014070 con 0x7f27981527f0 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 5 monclient: authenticate success, global_id 4513 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 monclient: _renew_subs 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27980c6490 con 0x7f27981527f0 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 1 -- 10.20.193.6:0/2471548596 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788002920 con 0x7f27981527f0 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f27967fc640 4 set_mon_vals no callback set 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f2794ff9640 10 monclient: dump: 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 client.4513.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 monclient: _renew_subs 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27980c7250 con 0x7f27981527f0 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 1 librados: init done 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.383+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27980747a0 con 0x7f27981527f0 2026-02-22T23:23:29.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.394+0000 7f2794ff9640 1 -- 10.20.193.6:0/2471548596 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2788019460 con 0x7f27981527f0 2026-02-22T23:23:29.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.395+0000 7f2794ff9640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2768078e50 0x7f276807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.395+0000 7f2797fff640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2768078e50 0x7f276807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.395+0000 7f2797fff640 10 monclient: get_auth_request con 0x7f2768078e50 auth_method 0 2026-02-22T23:23:29.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.396+0000 7f2794ff9640 10 client.4513.objecter ms_handle_connect 0x7f2768078e50 2026-02-22T23:23:29.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.396+0000 7f2797fff640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2768078e50 0x7f276807b310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2784004660 tx=0x7f2784009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.400+0000 7f969dffb640 1 -- 10.20.193.6:0/2095917272 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f96a0050030 con 0x7f96b006ff20 2026-02-22T23:23:29.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.400+0000 7f969dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.400+0000 7f969dffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.422+0000 7fbcad36a640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.422+0000 7fbcad36a640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.422+0000 7fbcad36a640 10 monclient: monmap: 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.422982+0000 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.422982+0000 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.423 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.422+0000 7fbcad36a640 1 Processor -- start 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.422+0000 7fbcad36a640 1 -- start start 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.422+0000 7fbcad36a640 10 monclient: init 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7f393f534640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7f393f534640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7f393f534640 10 monclient: monmap: 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.423714+0000 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.423714+0000 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.424 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient: _add_conns ranks=[4,0,3,5,2,1] 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fbca806ff20 0x7fbca8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient(hunting): picked mon.noname-b con 0x7fbca806ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbca80c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fbca815a5e0 0x7fbca814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient(hunting): picked mon.noname-a con 0x7fbca815a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbca80c7e90 con 0x7fbca80c99a0 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fbca8159a40 con 0x7fbca815a5e0 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbca80c8170 con 0x7fbca806ff20 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbcad36a640 10 monclient(hunting): authenticate will time out at 1084.150470s 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7f393f534640 1 Processor -- start 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7f393f534640 1 -- start start 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7f393f534640 10 monclient: init 2026-02-22T23:23:29.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.423+0000 7fbca77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fbca815a5e0 0x7fbca814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient: _add_conns ranks=[3,1,2,5,0,4] 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f39381527f0 0x7f3938073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient(hunting): picked mon.noname-a con 0x7f39381527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f393815d2c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f393815d730 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3938073c20 con 0x7f393815d2c0 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3938073f00 con 0x7f393815d730 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3938074b10 con 0x7f39381527f0 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393f534640 10 monclient(hunting): authenticate will time out at 1084.151271s 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393daaa640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f393815d730 0x7f39380ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36218/0 (socket says 10.20.193.6:36218) 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7f393daaa640 1 -- 10.20.193.6:0/3309402784 learned_addr learned my addr 10.20.193.6:0/3309402784 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7fbca77fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fbca815a5e0 0x7fbca814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39846/0 (socket says 10.20.193.6:39846) 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7fbca77fe640 1 -- 10.20.193.6:0/1673306858 learned_addr learned my addr 10.20.193.6:0/1673306858 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7fbca77fe640 10 monclient(hunting): get_auth_request con 0x7fbca815a5e0 auth_method 0 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7fbca77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7fbca77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.424+0000 7fbca77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7f393d2a9640 1 --2- 10.20.193.6:0/3309402784 >> v2:10.20.193.6:3300/0 conn(0x7f39381527f0 0x7f3938073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7fbca77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7fbca77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7fbca77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7f393d2a9640 10 monclient(hunting): get_auth_request con 0x7f39381527f0 auth_method 0 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7f393d2a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7f393d2a9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.425+0000 7f393d2a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 151596866 0 0) 0x7f3938073f00 con 0x7f393815d730 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7f3927fff640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7f3927fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7f3927fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7f3927fff640 10 monclient(hunting): my global_id is 4565 2026-02-22T23:23:29.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3920003620 con 0x7f393815d730 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 10 monclient(hunting): handle_auth_done global_id 4503 payload 274 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 1 -- 10.20.193.6:0/1673306858 >> v2:10.20.193.40:3300/0 conn(0x7fbca806ff20 msgr2=0x7fbca8070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 1 --2- 10.20.193.6:0/1673306858 >> v2:10.20.193.40:3300/0 conn(0x7fbca806ff20 0x7fbca8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 1 -- 10.20.193.6:0/1673306858 >> v1:10.20.193.6:6789/0 conn(0x7fbca80c99a0 legacy=0x7fbca8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.426+0000 7fbca77fe640 1 -- 10.20.193.6:0/1673306858 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca80c8450 con 0x7fbca815a5e0 2026-02-22T23:23:29.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 252816729 0 0) 0x7f3920003620 con 0x7f393815d730 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 >> v2:10.20.193.6:3300/0 conn(0x7f39381527f0 msgr2=0x7f3938073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 --2- 10.20.193.6:0/3309402784 >> v2:10.20.193.6:3300/0 conn(0x7f39381527f0 0x7f3938073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 >> v1:10.20.193.40:6789/0 conn(0x7f393815d2c0 legacy=0x7f393815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39380741e0 con 0x7f393815d730 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f3934003280 con 0x7f393815d730 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: dump: 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 >> v1:10.20.193.80:6789/0 conn(0x7f393815d730 legacy=0x7f39380ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3920003ba0 0x7f3920003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): picked mon.a con 0x7f3920003ba0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3920004900 0x7f3920004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): picked mon.b con 0x7f3920004900 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f392000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): picked mon.c con 0x7f39200054c0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3938074b10 con 0x7f3920003ba0 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f392000c380 con 0x7f3920004900 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f392000c580 con 0x7f39200054c0 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.427+0000 7f3927fff640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.428+0000 7f393daaa640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f392000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7f393daaa640 10 monclient(hunting): get_auth_request con 0x7f39200054c0 auth_method 0 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7f393daaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7f393daaa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7f393daaa640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7fbca77fe640 1 --2- 10.20.193.6:0/1673306858 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca814fe10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009e50 tx=0x7fbc9802f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc90ab7709b5e93 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7f393daaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7f393daaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.429+0000 7f393daaa640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 10 monclient(hunting): handle_auth_done global_id 4565 payload 293 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 1 -- 10.20.193.6:0/3309402784 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3920004900 msgr2=0x7f3920004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3920004900 0x7f3920004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 1 -- 10.20.193.6:0/3309402784 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3920003ba0 msgr2=0x7f3920003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3920003ba0 0x7f3920003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 1 monclient: found mon.c 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 1 -- 10.20.193.6:0/3309402784 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3920003620 con 0x7f39200054c0 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393f534640 5 monclient: authenticate success, global_id 4565 2026-02-22T23:23:29.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393f534640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.430+0000 7f393daaa640 1 -- 10.20.193.6:0/3309402784 reap_dead start 2026-02-22T23:23:29.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7f393daaa640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f392000bac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f393400a820 tx=0x7f393402fbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e099c14b9a7548a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 1 -- 10.20.193.6:0/1673306858 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9802fd60 con 0x7fbca815a5e0 2026-02-22T23:23:29.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:29.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: dump: 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 1 -- 10.20.193.6:0/1673306858 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc980373f0 con 0x7fbca815a5e0 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 1 -- 10.20.193.6:0/1673306858 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc980377c0 con 0x7fbca815a5e0 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbca5ffb640 10 monclient: dump: 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbcad36a640 5 monclient: authenticate success, global_id 4503 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbcad36a640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.431+0000 7fbcad36a640 4 set_mon_vals no callback set 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 10 monclient: shutdown 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 -- 10.20.193.6:0/1673306858 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 msgr2=0x7fbca814fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 --2- 10.20.193.6:0/1673306858 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca814fe10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009e50 tx=0x7fbc9802f2a0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 -- 10.20.193.6:0/1673306858 shutdown_connections 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 --2- 10.20.193.6:0/1673306858 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca814fe10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 --2- 10.20.193.6:0/1673306858 >> v2:10.20.193.40:3300/0 conn(0x7fbca806ff20 0x7fbca8070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 -- 10.20.193.6:0/1673306858 >> 10.20.193.6:0/1673306858 conn(0x7fbca80cd780 msgr2=0x7fbca80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 -- 10.20.193.6:0/1673306858 shutdown_connections 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 -- 10.20.193.6:0/1673306858 wait complete. 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 10 monclient: monmap: 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 librados: starting msgr at 2026-02-22T23:23:29.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 librados: starting objecter 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 Processor -- start 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 -- start start 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 librados: setting wanted keys 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 1 librados: calling monclient init 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7fbcad36a640 10 monclient: init 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393400a450 con 0x7f39200054c0 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 10 monclient: dump: 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39340316a0 con 0x7f39200054c0 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 1 -- 10.20.193.6:0/3309402784 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934031a90 con 0x7f39200054c0 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f3927fff640 10 monclient: dump: 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.435 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f393f534640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.432+0000 7f393f534640 4 set_mon_vals no callback set 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 10 monclient: shutdown 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 1 -- 10.20.193.6:0/3309402784 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 msgr2=0x7f392000bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f392000bac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f393400a820 tx=0x7f393402fbf0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbca806ff20 0x7fbca81f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): picked mon.noname-c con 0x7fbca806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbca80c99a0 0x7fbca81e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): picked mon.noname-b con 0x7fbca80c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): picked mon.noname-a con 0x7fbca815a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 1 -- 10.20.193.6:0/3309402784 shutdown_connections 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 1 --2- 10.20.193.6:0/3309402784 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f392000bac0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 1 -- 10.20.193.6:0/3309402784 >> 10.20.193.6:0/3309402784 conn(0x7f39380ce730 msgr2=0x7f39380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8150350 con 0x7fbca815a5e0 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbca80c8a10 con 0x7fbca80c99a0 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81504d0 con 0x7fbca806ff20 2026-02-22T23:23:29.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbcad36a640 10 monclient(hunting): authenticate will time out at 1084.160258s 2026-02-22T23:23:29.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 1 -- 10.20.193.6:0/3309402784 shutdown_connections 2026-02-22T23:23:29.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 1 -- 10.20.193.6:0/3309402784 wait complete. 2026-02-22T23:23:29.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca8157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39858/0 (socket says 10.20.193.6:39858) 2026-02-22T23:23:29.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 1 -- 10.20.193.6:0/2129818887 learned_addr learned my addr 10.20.193.6:0/2129818887 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 10 monclient(hunting): get_auth_request con 0x7fbca815a5e0 auth_method 0 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca6ffd640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbca806ff20 0x7fbca81f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7f393f534640 10 monclient: monmap: 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.439 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca6ffd640 10 monclient(hunting): get_auth_request con 0x7fbca806ff20 auth_method 0 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.433+0000 7fbca77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 librados: starting msgr at 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 librados: starting objecter 2026-02-22T23:23:29.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 Processor -- start 2026-02-22T23:23:29.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7fbca6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7fbca6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7fbca6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 -- start start 2026-02-22T23:23:29.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 librados: setting wanted keys 2026-02-22T23:23:29.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 librados: calling monclient init 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient: init 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f3938157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): picked mon.noname-c con 0x7f39200054c0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f39381f4370 0x7f393815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): picked mon.noname-b con 0x7f39381f4370 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f393815ed40 0x7f393814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): picked mon.noname-a con 0x7f393815ed40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3938074fa0 con 0x7f393815ed40 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f39380747a0 con 0x7f39381f4370 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3938075120 con 0x7f39200054c0 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.434+0000 7f393f534640 10 monclient(hunting): authenticate will time out at 1084.161553s 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 10 monclient(hunting): handle_auth_done global_id 4568 payload 824 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 1 -- 10.20.193.6:0/2129818887 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbca80c99a0 msgr2=0x7fbca81e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbca80c99a0 0x7fbca81e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 1 -- 10.20.193.6:0/2129818887 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 msgr2=0x7fbca8157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca8157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.435+0000 7fbca6ffd640 1 -- 10.20.193.6:0/2129818887 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca81f15c0 con 0x7fbca806ff20 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.436+0000 7fbca6ffd640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbca806ff20 0x7fbca81f1d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b580 tx=0x7fbc9c00ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.436+0000 7fbc877fe640 10 client.?.objecter ms_handle_connect 0x7fbca806ff20 2026-02-22T23:23:29.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.436+0000 7fbc877fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.436+0000 7f393daaa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f393815ed40 0x7f393814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f393815ed40 0x7f393814beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39874/0 (socket says 10.20.193.6:39874) 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 1 -- 10.20.193.6:0/1311347986 learned_addr learned my addr 10.20.193.6:0/1311347986 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 10 monclient(hunting): get_auth_request con 0x7f393815ed40 auth_method 0 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393daaa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393d2a9640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f3938157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 1 -- 10.20.193.6:0/2129818887 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c004480 con 0x7fbca806ff20 2026-02-22T23:23:29.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: dump: 2026-02-22T23:23:29.445 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.445 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.445 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 1 -- 10.20.193.6:0/2129818887 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc9c014070 con 0x7fbca806ff20 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 1 -- 10.20.193.6:0/2129818887 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c002920 con 0x7fbca806ff20 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 5 monclient: authenticate success, global_id 4568 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbca57fa640 4 set_mon_vals no callback set 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 10 monclient: _renew_subs 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca81f4ff0 con 0x7fbca806ff20 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbc877fe640 10 monclient: dump: 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.446 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 10 client.4568.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 10 monclient: _renew_subs 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca81f55b0 con 0x7fbca806ff20 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393d2a9640 10 monclient(hunting): get_auth_request con 0x7f39200054c0 auth_method 0 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393d2a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393d2a9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7f393d2a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.437+0000 7fbcad36a640 1 librados: init done 2026-02-22T23:23:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393d2a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393d2a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393d2a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7fbcad36a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7fbcad36a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7fbcad36a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca8150860 con 0x7fbca806ff20 2026-02-22T23:23:29.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 10 monclient(hunting): handle_auth_done global_id 4512 payload 824 2026-02-22T23:23:29.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 1 -- 10.20.193.6:0/1311347986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 msgr2=0x7f3938157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f3938157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 1 -- 10.20.193.6:0/1311347986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f39381f4370 msgr2=0x7f393815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f39381f4370 0x7f393815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.438+0000 7f393daaa640 1 -- 10.20.193.6:0/1311347986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3938156b20 con 0x7f393815ed40 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.439+0000 7f393daaa640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f393815ed40 0x7f393814beb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f393400a950 tx=0x7f39381601e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.439+0000 7f3925ffb640 10 client.?.objecter ms_handle_connect 0x7f393815ed40 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.439+0000 7f3925ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 1 -- 10.20.193.6:0/1311347986 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39340037a0 con 0x7f393815ed40 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: dump: 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 1 -- 10.20.193.6:0/1311347986 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393403f070 con 0x7f393815ed40 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 1 -- 10.20.193.6:0/1311347986 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393403b660 con 0x7f393815ed40 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 5 monclient: authenticate success, global_id 4512 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 10 monclient: _renew_subs 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f39277fe640 4 set_mon_vals no callback set 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3938156e00 con 0x7f393815ed40 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f3925ffb640 10 monclient: dump: 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 10 client.4512.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 10 monclient: _renew_subs 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39381573c0 con 0x7f393815ed40 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.441+0000 7f393f534640 1 librados: init done 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.442+0000 7f393f534640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.442+0000 7f393f534640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.442+0000 7f393f534640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.442+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3938152bd0 con 0x7f393815ed40 2026-02-22T23:23:29.452 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-22T23:23:29.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.458+0000 7f695263c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-22T23:23:29.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.458+0000 7f695263c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-22T23:23:29.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.458+0000 7f695263c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.458+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f694c1502b0 con 0x7f694c14eec0 2026-02-22T23:23:29.465 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:23:29.466 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-22T23:23:29.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.485+0000 7f96b7534640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-22T23:23:29.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.485+0000 7f96b7534640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-22T23:23:29.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.485+0000 7f96b7534640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.485+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 14} v 0) -- 0x7f96b0153550 con 0x7f96b006ff20 2026-02-22T23:23:29.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.490+0000 7f2794ff9640 1 -- 10.20.193.6:0/2471548596 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f278805dd80 con 0x7f27981527f0 2026-02-22T23:23:29.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.490+0000 7f2794ff9640 10 client.4513.objecter ms_dispatch2 0x7f2798000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.490+0000 7f2794ff9640 3 client.4513.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.490+0000 7f2794ff9640 3 client.4513.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.490+0000 7f2794ff9640 20 client.4513.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.497+0000 7f2794ff9640 1 -- 10.20.193.6:0/2471548596 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2788062440 con 0x7f27981527f0 2026-02-22T23:23:29.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.497+0000 7f2794ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.497+0000 7f2794ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.501+0000 7f6948ff9640 1 -- 10.20.193.6:0/2215204681 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f693c061340 con 0x7f694c14eec0 2026-02-22T23:23:29.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.501+0000 7f6948ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 9}] 2026-02-22T23:23:29.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.501+0000 7f6948ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.503 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 10 librados: watch_flush enter 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 10 librados: watch_flush exit 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 20 client.4507.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 10 client.4507.objecter successfully canceled tick 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f691c078e30 msgr2=0x7f691c07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f691c078e30 0x7f691c07b2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6938007cc0 tx=0x7f6938002870 comp rx=0 tx=0).stop 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 10 monclient: shutdown 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f694c14eec0 msgr2=0x7f694c1f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f694c14eec0 0x7f694c1f2a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f693c004f30 tx=0x7f693c007500 comp rx=0 tx=0).stop 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 shutdown_connections 2026-02-22T23:23:29.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f691c078e30 0x7f691c07b2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f694c1f44d0 0x7f694c1f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f694c14eec0 0x7f694c1f2a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 --2- 10.20.193.6:0/2215204681 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6930009ab0 0x7f694c14e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 >> 10.20.193.6:0/2215204681 conn(0x7f694c0ce730 msgr2=0x7f694c0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 shutdown_connections 2026-02-22T23:23:29.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 -- 10.20.193.6:0/2215204681 wait complete. 2026-02-22T23:23:29.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.504+0000 7f695263c640 1 librados: shutdown 2026-02-22T23:23:29.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.9 2026-02-22T23:23:29.515 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.518+0000 7f7e92aa0640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.518+0000 7f7e92aa0640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.518+0000 7f7e92aa0640 10 monclient: monmap: 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.519086+0000 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.519086+0000 2026-02-22T23:23:29.520 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.518+0000 7f7e92aa0640 1 Processor -- start 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.518+0000 7f7e92aa0640 1 -- start start 2026-02-22T23:23:29.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.518+0000 7f7e92aa0640 10 monclient: init 2026-02-22T23:23:29.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient: _add_conns ranks=[4,0,1,3,5,2] 2026-02-22T23:23:29.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7e8c06ff20 0x7f7e8c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient(hunting): picked mon.noname-b con 0x7f7e8c06ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7e8c0c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:29.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7e8c15a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c0c7e90 con 0x7f7e8c0c99a0 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c0c8170 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7e8c159a40 con 0x7f7e8c06ff20 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e92aa0640 10 monclient(hunting): authenticate will time out at 1084.246454s 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.519+0000 7f7e90815640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7e8c06ff20 0x7f7e8c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7e8c06ff20 0x7f7e8c070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36574/0 (socket says 10.20.193.6:36574) 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 1 -- 10.20.193.6:0/3203586426 learned_addr learned my addr 10.20.193.6:0/3203586426 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 10 monclient(hunting): get_auth_request con 0x7f7e8c06ff20 auth_method 0 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.520+0000 7f7e90815640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 10 monclient(hunting): handle_auth_done global_id 4516 payload 274 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 1 -- 10.20.193.6:0/3203586426 >> v1:10.20.193.40:6789/0 conn(0x7f7e8c15a5e0 legacy=0x7f7e8c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 1 -- 10.20.193.6:0/3203586426 >> v1:10.20.193.6:6789/0 conn(0x7f7e8c0c99a0 legacy=0x7f7e8c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.521+0000 7f7e90815640 1 -- 10.20.193.6:0/3203586426 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c0c8450 con 0x7f7e8c06ff20 2026-02-22T23:23:29.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.523+0000 7f7e90815640 1 --2- 10.20.193.6:0/3203586426 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c06ff20 0x7f7e8c070300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0099a0 tx=0x7f7e7c02ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4c32e16011d311 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.524+0000 7f7e837fe640 1 -- 10.20.193.6:0/3203586426 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c03c070 con 0x7f7e8c06ff20 2026-02-22T23:23:29.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: dump: 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 1 -- 10.20.193.6:0/3203586426 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7c02fbd0 con 0x7f7e8c06ff20 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 1 -- 10.20.193.6:0/3203586426 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c037440 con 0x7f7e8c06ff20 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 5 monclient: authenticate success, global_id 4516 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e837fe640 10 monclient: dump: 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.529 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 4 set_mon_vals no callback set 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 10 monclient: shutdown 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 1 -- 10.20.193.6:0/3203586426 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c06ff20 msgr2=0x7f7e8c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/3203586426 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c06ff20 0x7f7e8c070300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0099a0 tx=0x7f7e7c02ef50 comp rx=0 tx=0).stop 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 1 -- 10.20.193.6:0/3203586426 shutdown_connections 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/3203586426 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c06ff20 0x7f7e8c070300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 1 -- 10.20.193.6:0/3203586426 >> 10.20.193.6:0/3203586426 conn(0x7f7e8c0cd780 msgr2=0x7f7e8c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 1 -- 10.20.193.6:0/3203586426 shutdown_connections 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 1 -- 10.20.193.6:0/3203586426 wait complete. 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.525+0000 7f7e92aa0640 10 monclient: monmap: 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 librados: starting msgr at 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 librados: starting objecter 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 Processor -- start 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 -- start start 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 librados: setting wanted keys 2026-02-22T23:23:29.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 librados: calling monclient init 2026-02-22T23:23:29.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient: init 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7e8c06ff20 0x7f7e8c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): picked mon.noname-a con 0x7f7e8c06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7e8c0c99a0 0x7f7e8c1f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): picked mon.noname-c con 0x7f7e8c0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c15a5e0 0x7f7e8c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): picked mon.noname-b con 0x7f7e8c15a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c150520 con 0x7f7e8c06ff20 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c0c8a10 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1506a0 con 0x7f7e8c0c99a0 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.526+0000 7f7e92aa0640 10 monclient(hunting): authenticate will time out at 1084.253677s 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c15a5e0 0x7f7e8c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c15a5e0 0x7f7e8c06e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36586/0 (socket says 10.20.193.6:36586) 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 1 -- 10.20.193.6:0/2712884179 learned_addr learned my addr 10.20.193.6:0/2712884179 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 10 monclient(hunting): get_auth_request con 0x7f7e8c15a5e0 auth_method 0 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.527+0000 7f7e91016640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 10 monclient(hunting): handle_auth_done global_id 4522 payload 824 2026-02-22T23:23:29.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 1 -- 10.20.193.6:0/2712884179 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7e8c0c99a0 msgr2=0x7f7e8c1f64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7e8c0c99a0 0x7f7e8c1f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 1 -- 10.20.193.6:0/2712884179 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7e8c06ff20 msgr2=0x7f7e8c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7e8c06ff20 0x7f7e8c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.528+0000 7f7e91016640 1 -- 10.20.193.6:0/2712884179 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c155390 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.530+0000 7f7e91016640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c15a5e0 0x7f7e8c06e4a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800cba0 tx=0x7f7e8800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.530+0000 7f7e817fa640 10 client.?.objecter ms_handle_connect 0x7f7e8c15a5e0 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.530+0000 7f7e817fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 1 -- 10.20.193.6:0/2712884179 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800ad50 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: dump: 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.536 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 1 -- 10.20.193.6:0/2712884179 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e88003a70 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 5 monclient: authenticate success, global_id 4522 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 10 monclient: _renew_subs 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c155670 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 1 -- 10.20.193.6:0/2712884179 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880050d0 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e817fa640 10 monclient: dump: 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e82ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 10 client.4522.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 10 monclient: _renew_subs 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c155c30 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.531+0000 7f7e92aa0640 1 librados: init done 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f59e5ac4640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f59e5ac4640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f59e5ac4640 10 monclient: monmap: 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.532665+0000 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.532665+0000 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.539 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f7e92aa0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f7e92aa0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f7e92aa0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8c070300 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f59e5ac4640 1 Processor -- start 2026-02-22T23:23:29.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f76a13e4640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f76a13e4640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f76a13e4640 10 monclient: monmap: 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.533200+0000 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.533200+0000 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f59e5ac4640 1 -- start start 2026-02-22T23:23:29.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.532+0000 7f59e5ac4640 10 monclient: init 2026-02-22T23:23:29.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 1 Processor -- start 2026-02-22T23:23:29.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 1 -- start start 2026-02-22T23:23:29.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient: init 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient: _add_conns ranks=[1,3,5,4,0,2] 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f59e01527f0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f59e0073530 0x7f59e015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient(hunting): picked mon.noname-a con 0x7f59e0073530 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f59e015f900 0x7f59e00ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient(hunting): picked mon.noname-c con 0x7f59e015f900 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59e00739a0 con 0x7f59e01527f0 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f59e0074b10 con 0x7f59e0073530 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f59e015fd90 con 0x7f59e015f900 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f59e5ac4640 10 monclient(hunting): authenticate will time out at 1084.260359s 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient: _add_conns ranks=[1,2,0,4,5,3] 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f769c06ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f769c0c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f769c15a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769c0c7e90 con 0x7f769c15a5e0 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769c0c8170 con 0x7f769c06ff20 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f769c0c8450 con 0x7f769c0c99a0 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.533+0000 7f76a13e4640 10 monclient(hunting): authenticate will time out at 1084.260697s 2026-02-22T23:23:29.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.535+0000 7fbc877fe640 1 -- 10.20.193.6:0/2129818887 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fbc9c002ac0 con 0x7fbca806ff20 2026-02-22T23:23:29.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.536+0000 7fbc877fe640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbc74078e00 0x7fbc7407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.536+0000 7fbca67fc640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbc74078e00 0x7fbc7407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.536+0000 7fbca67fc640 10 monclient: get_auth_request con 0x7fbc74078e00 auth_method 0 2026-02-22T23:23:29.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.537+0000 7fbca67fc640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbc74078e00 0x7fbc7407b2c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbc90002770 tx=0x7fbc90007ad0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.537+0000 7fbc877fe640 10 client.4568.objecter ms_handle_connect 0x7fbc74078e00 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.543+0000 7f7e817fa640 1 -- 10.20.193.6:0/2712884179 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7e88005270 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.543+0000 7f7e817fa640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7e5c078e00 0x7f7e5c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.543+0000 7f7e90815640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7e5c078e00 0x7f7e5c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.543+0000 7f7e90815640 10 monclient: get_auth_request con 0x7f7e5c078e00 auth_method 0 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.544+0000 7f7e90815640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7e5c078e00 0x7f7e5c07b2c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009970 tx=0x7f7e7c031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.544+0000 7f7e817fa640 10 client.4522.objecter ms_handle_connect 0x7f7e5c078e00 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.544+0000 7f7e817fa640 1 -- 10.20.193.6:0/2712884179 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f7e8805e710 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.544+0000 7f7e817fa640 10 client.4522.objecter ms_dispatch2 0x7f7e8c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.544+0000 7f7e817fa640 3 client.4522.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.544+0000 7f7e817fa640 3 client.4522.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.544+0000 7f7e817fa640 20 client.4522.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7f59df7fe640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f59e01527f0 0x7f59e015d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60324/0 (socket says 10.20.193.6:60324) 2026-02-22T23:23:29.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7f59df7fe640 1 -- 10.20.193.6:0/2584265875 learned_addr learned my addr 10.20.193.6:0/2584265875 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7fbc877fe640 1 -- 10.20.193.6:0/2129818887 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fbc9c05df60 con 0x7fbca806ff20 2026-02-22T23:23:29.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7fbc877fe640 10 client.4568.objecter ms_dispatch2 0x7fbca8000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7fbc877fe640 3 client.4568.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7fbc877fe640 3 client.4568.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7fbc877fe640 20 client.4568.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.545+0000 7f3925ffb640 1 -- 10.20.193.6:0/1311347986 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f393404e060 con 0x7f393815ed40 2026-02-22T23:23:29.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.546+0000 7f3925ffb640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3908078e50 0x7f390807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.546+0000 7f393d2a9640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3908078e50 0x7f390807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.546+0000 7f393d2a9640 10 monclient: get_auth_request con 0x7f3908078e50 auth_method 0 2026-02-22T23:23:29.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.546+0000 7f3925ffb640 10 client.4512.objecter ms_handle_connect 0x7f3908078e50 2026-02-22T23:23:29.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.546+0000 7f393d2a9640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3908078e50 0x7f390807b310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39280048b0 tx=0x7f3928008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.552+0000 7f969dffb640 1 -- 10.20.193.6:0/2095917272 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 14}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f96b0153550 con 0x7f96b006ff20 2026-02-22T23:23:29.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.552+0000 7f969dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 14}] 2026-02-22T23:23:29.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.552+0000 7f969dffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.553 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:29.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.552+0000 7fbc877fe640 1 -- 10.20.193.6:0/2129818887 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbc9c062620 con 0x7fbca806ff20 2026-02-22T23:23:29.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.552+0000 7fbc877fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.552+0000 7fbc877fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 10 librados: watch_flush enter 2026-02-22T23:23:29.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 10 librados: watch_flush exit 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 20 client.4562.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 10 client.4562.objecter successfully canceled tick 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 1 -- 10.20.193.6:0/1311347986 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f393400a510 con 0x7f393815ed40 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 10 client.4512.objecter ms_dispatch2 0x7f3938000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 3 client.4512.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 3 client.4512.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9680078e30 msgr2=0x7f968007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9680078e30 0x7f968007b2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96b00c5ce0 tx=0x7f96a4006d90 comp rx=0 tx=0).stop 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 10 monclient: shutdown 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 msgr2=0x7f96b014cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b014cd70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f96a0036000 tx=0x7f96a00047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 20 client.4512.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 shutdown_connections 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9680078e30 0x7f968007b2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f96b015a410 0x7f96b0153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 1 -- 10.20.193.6:0/1311347986 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3934036030 con 0x7f393815ed40 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f96b00c99a0 0x7f96b0152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 --2- 10.20.193.6:0/2095917272 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f96b006ff20 0x7f96b014cd70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f3925ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 >> 10.20.193.6:0/2095917272 conn(0x7f96b00cd780 msgr2=0x7f96b0157840 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 shutdown_connections 2026-02-22T23:23:29.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.554+0000 7f96b7534640 1 -- 10.20.193.6:0/2095917272 wait complete. 2026-02-22T23:23:29.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f96b7534640 1 librados: shutdown 2026-02-22T23:23:29.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 1 --2- 10.20.193.6:0/2584265875 >> v2:10.20.193.80:3300/0 conn(0x7f59e015f900 0x7f59e00ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f769a7fc640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f769c0c99a0 0x7f769c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36230/0 (socket says 10.20.193.6:36230) 2026-02-22T23:23:29.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f769a7fc640 1 -- 10.20.193.6:0/2372577386 learned_addr learned my addr 10.20.193.6:0/2372577386 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 10 monclient(hunting): get_auth_request con 0x7f59e015f900 auth_method 0 2026-02-22T23:23:29.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.555+0000 7f59dffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 10 monclient(hunting): handle_auth_done global_id 4574 payload 274 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 1 -- 10.20.193.6:0/2584265875 >> v2:10.20.193.6:3300/0 conn(0x7f59e0073530 msgr2=0x7f59e015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 1 --2- 10.20.193.6:0/2584265875 >> v2:10.20.193.6:3300/0 conn(0x7f59e0073530 0x7f59e015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 1 -- 10.20.193.6:0/2584265875 >> v1:10.20.193.40:6789/0 conn(0x7f59e01527f0 legacy=0x7f59e015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.556+0000 7f59dffff640 1 -- 10.20.193.6:0/2584265875 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e015d9d0 con 0x7f59e015f900 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.558+0000 7f59dffff640 1 --2- 10.20.193.6:0/2584265875 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e015f900 0x7f59e00ccf70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59e006cdf0 tx=0x7f59d402f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930e2dc7c388f633 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.558+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145637453 0 0) 0x7f769c0c8450 con 0x7f769c0c99a0 2026-02-22T23:23:29.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.558+0000 7f7699ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.558+0000 7f7699ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.558+0000 7f7699ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.558+0000 7f7699ffb640 10 monclient(hunting): my global_id is 4577 2026-02-22T23:23:29.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.558+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7688003620 con 0x7f769c0c99a0 2026-02-22T23:23:29.561 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-22T23:23:29.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 1 -- 10.20.193.6:0/2584265875 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d403c070 con 0x7f59e015f900 2026-02-22T23:23:29.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: dump: 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 1 -- 10.20.193.6:0/2584265875 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d402fe30 con 0x7f59e015f900 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 1 -- 10.20.193.6:0/2584265875 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4037640 con 0x7f59e015f900 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59e5ac4640 5 monclient: authenticate success, global_id 4574 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59e5ac4640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59e5ac4640 4 set_mon_vals no callback set 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.561+0000 7f59de7fc640 10 monclient: dump: 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 10 monclient: shutdown 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 -- 10.20.193.6:0/2584265875 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e015f900 msgr2=0x7f59e00ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/2584265875 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e015f900 0x7f59e00ccf70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59e006cdf0 tx=0x7f59d402f1c0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 -- 10.20.193.6:0/2584265875 shutdown_connections 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/2584265875 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e015f900 0x7f59e00ccf70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/2584265875 >> v2:10.20.193.6:3300/0 conn(0x7f59e0073530 0x7f59e015f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 -- 10.20.193.6:0/2584265875 >> 10.20.193.6:0/2584265875 conn(0x7f59e00ce730 msgr2=0x7f59e00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 -- 10.20.193.6:0/2584265875 shutdown_connections 2026-02-22T23:23:29.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 -- 10.20.193.6:0/2584265875 wait complete. 2026-02-22T23:23:29.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2655989726 0 0) 0x7f7688003620 con 0x7f769c0c99a0 2026-02-22T23:23:29.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 >> v1:10.20.193.40:6789/0 conn(0x7f769c06ff20 legacy=0x7f769c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 >> v1:10.20.193.6:6789/0 conn(0x7f769c15a5e0 legacy=0x7f769c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c0c8730 con 0x7f769c0c99a0 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f7684002f00 con 0x7f769c0c99a0 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: dump: 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 >> v1:10.20.193.80:6789/0 conn(0x7f769c0c99a0 legacy=0x7f769c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7688003b80 0x7f7688003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): picked mon.a con 0x7f7688003b80 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7688004900 0x7f7688004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): picked mon.b con 0x7f7688004900 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7688005510 0x7f768800bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): picked mon.c con 0x7f7688005510 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f768800c3f0 con 0x7f7688003b80 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f768800c5a0 con 0x7f7688004900 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f768800c7a0 con 0x7f7688005510 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f7699ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 10 monclient: monmap: 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.569 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 librados: starting msgr at 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.562+0000 7f59e5ac4640 1 librados: starting objecter 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 Processor -- start 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 -- start start 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 librados: setting wanted keys 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 librados: calling monclient init 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient: init 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59e0073530 0x7f59e00c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): picked mon.noname-a con 0x7f59e0073530 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e01527f0 0x7f59e01f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): picked mon.noname-c con 0x7f59e01527f0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59e015f900 0x7f59e00c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): picked mon.noname-b con 0x7f59e015f900 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f59e00741e0 con 0x7f59e0073530 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f59e0073f00 con 0x7f59e015f900 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f59e00744c0 con 0x7f59e01527f0 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.563+0000 7f59e5ac4640 10 monclient(hunting): authenticate will time out at 1084.290502s 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7688005510 0x7f768800bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 10 monclient(hunting): get_auth_request con 0x7f7688005510 auth_method 0 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f59deffd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e01527f0 0x7f59e01f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.564+0000 7f769b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f59deffd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e01527f0 0x7f59e01f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58698/0 (socket says 10.20.193.6:58698) 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f59deffd640 1 -- 10.20.193.6:0/1300932326 learned_addr learned my addr 10.20.193.6:0/1300932326 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f59deffd640 10 monclient(hunting): get_auth_request con 0x7f59e01527f0 auth_method 0 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f59deffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f59deffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f59deffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.14 2026-02-22T23:23:29.573 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:29.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f769b7fe640 10 monclient(hunting): handle_auth_done global_id 4577 payload 293 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f769b7fe640 1 -- 10.20.193.6:0/2372577386 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7688004900 msgr2=0x7f7688004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f769b7fe640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7688004900 0x7f7688004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f769b7fe640 1 -- 10.20.193.6:0/2372577386 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7688003b80 msgr2=0x7f7688003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f769b7fe640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7688003b80 0x7f7688003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f769b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.565+0000 7f769b7fe640 1 monclient: found mon.c 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f769b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f769b7fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f769b7fe640 1 -- 10.20.193.6:0/2372577386 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7688003620 con 0x7f7688005510 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f769b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f769b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f769b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f76a13e4640 5 monclient: authenticate success, global_id 4577 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f76a13e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f59deffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f59deffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f59deffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.566+0000 7f769b7fe640 1 -- 10.20.193.6:0/2372577386 reap_dead start 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 10 monclient(hunting): handle_auth_done global_id 4580 payload 824 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 1 -- 10.20.193.6:0/1300932326 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59e015f900 msgr2=0x7f59e00c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59e015f900 0x7f59e00c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 1 -- 10.20.193.6:0/1300932326 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59e0073530 msgr2=0x7f59e00c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59e0073530 0x7f59e00c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f59deffd640 1 -- 10.20.193.6:0/1300932326 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e00c6f80 con 0x7f59e01527f0 2026-02-22T23:23:29.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.568+0000 7f769b7fe640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7688005510 0x7f768800bb40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f768c004a50 tx=0x7f768c02f3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182f6e9ec351b16c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.569+0000 7f59deffd640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e01527f0 0x7f59e01f4950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59d000b780 tx=0x7f59d000bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.569+0000 7f59bbfff640 10 client.?.objecter ms_handle_connect 0x7f59e01527f0 2026-02-22T23:23:29.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.569+0000 7f59bbfff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.576 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c004620 con 0x7f7688005510 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 10 monclient: dump: 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.577 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c02fe40 con 0x7f7688005510 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 1 -- 10.20.193.6:0/2372577386 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c037680 con 0x7f7688005510 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 4 set_mon_vals no callback set 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f7699ffb640 10 monclient: dump: 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 10 monclient: shutdown 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 1 -- 10.20.193.6:0/2372577386 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7688005510 msgr2=0x7f768800bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7688005510 0x7f768800bb40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f768c004a50 tx=0x7f768c02f3e0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 1 -- 10.20.193.6:0/2372577386 shutdown_connections 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 1 --2- 10.20.193.6:0/2372577386 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7688005510 0x7f768800bb40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 1 -- 10.20.193.6:0/2372577386 >> 10.20.193.6:0/2372577386 conn(0x7f769c0cd780 msgr2=0x7f769c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 1 -- 10.20.193.6:0/2372577386 shutdown_connections 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f76a13e4640 1 -- 10.20.193.6:0/2372577386 wait complete. 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 1 -- 10.20.193.6:0/1300932326 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0004480 con 0x7f59e01527f0 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 10 monclient: dump: 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59bbfff640 1 -- 10.20.193.6:0/1300932326 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59d0010070 con 0x7f59e01527f0 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59e5ac4640 5 monclient: authenticate success, global_id 4580 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.572+0000 7f59e5ac4640 10 monclient: _renew_subs 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e00c7260 con 0x7f59e01527f0 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59bbfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59bbfff640 1 -- 10.20.193.6:0/1300932326 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0002920 con 0x7f59e01527f0 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59bbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59ddffb640 4 set_mon_vals no callback set 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59bbfff640 10 monclient: dump: 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 10 client.4580.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 10 monclient: _renew_subs 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e00c7820 con 0x7f59e01527f0 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 1 librados: init done 2026-02-22T23:23:29.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 10 monclient: monmap: 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 1 librados: starting msgr at 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 1 librados: starting objecter 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 1 Processor -- start 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 1 -- start start 2026-02-22T23:23:29.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 1 librados: setting wanted keys 2026-02-22T23:23:29.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 1 librados: calling monclient init 2026-02-22T23:23:29.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 10 monclient: init 2026-02-22T23:23:29.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e00cc640 con 0x7f59e01527f0 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7688005510 0x7f769c1f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.573+0000 7f76a13e4640 10 monclient(hunting): picked mon.noname-b con 0x7f7688005510 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f769c156f60 0x7f769c06e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 10 monclient(hunting): picked mon.noname-a con 0x7f769c156f60 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f769c06e750 0x7f769c1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 10 monclient(hunting): picked mon.noname-c con 0x7f769c06e750 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f769c070300 con 0x7f769c156f60 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f769c159a40 con 0x7f7688005510 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f769c070480 con 0x7f769c06e750 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.574+0000 7f76a13e4640 10 monclient(hunting): authenticate will time out at 1084.301029s 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.582+0000 7f769a7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f769c156f60 0x7f769c06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.582+0000 7f769a7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f769c156f60 0x7f769c06e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39912/0 (socket says 10.20.193.6:39912) 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.582+0000 7f769a7fc640 1 -- 10.20.193.6:0/3989744301 learned_addr learned my addr 10.20.193.6:0/3989744301 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.582+0000 7f769a7fc640 10 monclient(hunting): get_auth_request con 0x7f769c156f60 auth_method 0 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.582+0000 7f769a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.582+0000 7f769a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.582+0000 7f769a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.583+0000 7f769a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.583+0000 7f769a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.583+0000 7f769a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 10 monclient(hunting): handle_auth_done global_id 4518 payload 824 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 1 -- 10.20.193.6:0/3989744301 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f769c06e750 msgr2=0x7f769c1f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f769c06e750 0x7f769c1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 1 -- 10.20.193.6:0/3989744301 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7688005510 msgr2=0x7f769c1f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7688005510 0x7f769c1f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.584+0000 7f769a7fc640 1 -- 10.20.193.6:0/3989744301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c154340 con 0x7f769c156f60 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.585+0000 7f279e2ea640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.585+0000 7f279e2ea640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.585+0000 7f279e2ea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.585+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f279815e200 con 0x7f27981527f0 2026-02-22T23:23:29.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.596+0000 7f769a7fc640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f769c156f60 0x7f769c06e210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f768400ca70 tx=0x7f768400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.596+0000 7f767b7fe640 10 client.?.objecter ms_handle_connect 0x7f769c156f60 2026-02-22T23:23:29.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.596+0000 7f767b7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.628+0000 7f23d53e4640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.628+0000 7f23d53e4640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.628+0000 7f23d53e4640 10 monclient: monmap: 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.629418+0000 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.629418+0000 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.630 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 1 Processor -- start 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 1 -- start start 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient: init 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient: _add_conns ranks=[3,1,2,4,5,0] 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f23d006ff20 0x7f23d0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient(hunting): picked mon.noname-a con 0x7f23d006ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:29.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f23d00c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f23d015a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d00c7e90 con 0x7f23d00c99a0 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d00c8170 con 0x7f23d015a5e0 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f23d0159a40 con 0x7f23d006ff20 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.629+0000 7f23d53e4640 10 monclient(hunting): authenticate will time out at 1084.356812s 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.630+0000 7f23ceffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f23d006ff20 0x7f23d0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.630+0000 7f23ceffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f23d006ff20 0x7f23d0070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39916/0 (socket says 10.20.193.6:39916) 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.630+0000 7f23ceffd640 1 -- 10.20.193.6:0/3630709713 learned_addr learned my addr 10.20.193.6:0/3630709713 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.630+0000 7f23ceffd640 10 monclient(hunting): get_auth_request con 0x7f23d006ff20 auth_method 0 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.630+0000 7f23ceffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.630+0000 7f23ceffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.630+0000 7f23ceffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.631+0000 7f23ceffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.631+0000 7f23ceffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.631+0000 7f23ceffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.633+0000 7f23ceffd640 10 monclient(hunting): handle_auth_done global_id 4521 payload 274 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.634+0000 7f23ceffd640 1 -- 10.20.193.6:0/3630709713 >> v1:10.20.193.80:6789/0 conn(0x7f23d015a5e0 legacy=0x7f23d014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.634+0000 7f23ceffd640 1 -- 10.20.193.6:0/3630709713 >> v1:10.20.193.40:6789/0 conn(0x7f23d00c99a0 legacy=0x7f23d0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.634+0000 7f23ceffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.634+0000 7f23ceffd640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.634+0000 7f23ceffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.634+0000 7f23ceffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.634+0000 7f23ceffd640 1 -- 10.20.193.6:0/3630709713 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d00c8450 con 0x7f23d006ff20 2026-02-22T23:23:29.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.635+0000 7f23ceffd640 1 --2- 10.20.193.6:0/3630709713 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d006ff20 0x7f23d0070300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f23c40099a0 tx=0x7f23c402ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8369c7c11e0702c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.642+0000 7f393f534640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:29.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.642+0000 7f393f534640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:29.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.642+0000 7f393f534640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.642+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f393806c820 con 0x7f393815ed40 2026-02-22T23:23:29.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.643+0000 7fbcad36a640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:29.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.643+0000 7fbcad36a640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:29.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.643+0000 7fbcad36a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.643+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbca80c4830 con 0x7fbca806ff20 2026-02-22T23:23:29.646 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T23:23:29.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.646+0000 7f7e817fa640 1 -- 10.20.193.6:0/2712884179 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7e880102e0 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.646+0000 7f7e817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.646+0000 7f7e817fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.648+0000 7fafefd1b640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.648+0000 7fafefd1b640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.648+0000 7fafefd1b640 10 monclient: monmap: 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.649397+0000 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.649397+0000 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.650 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7f2794ff9640 1 -- 10.20.193.6:0/2471548596 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2788061d40 con 0x7f27981527f0 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7f2794ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2026-02-22T23:23:29.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7f2794ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.652 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:29.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 1 Processor -- start 2026-02-22T23:23:29.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 1 -- start start 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient: init 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient: _add_conns ranks=[1,5,4,2,0,3] 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fafe815f650 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fafe815fa30 0x7fafe80d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient(hunting): picked mon.noname-c con 0x7fafe815fa30 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fafe80d1ea0 0x7fafe80c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient(hunting): picked mon.noname-b con 0x7fafe80d1ea0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe80d2450 con 0x7fafe815f650 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fafe80c8dd0 con 0x7fafe80d1ea0 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fafe80d2730 con 0x7fafe815fa30 2026-02-22T23:23:29.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.649+0000 7fafefd1b640 10 monclient(hunting): authenticate will time out at 1084.376854s 2026-02-22T23:23:29.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 10 librados: watch_flush enter 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 10 librados: watch_flush exit 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 20 client.4513.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 10 client.4513.objecter successfully canceled tick 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2768078e50 msgr2=0x7f276807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2768078e50 0x7f276807b310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2784004660 tx=0x7f2784009110 comp rx=0 tx=0).stop 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 10 monclient: shutdown 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f27981527f0 msgr2=0x7f27980cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f27981527f0 0x7f27980cd170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f278800b500 tx=0x7f278800b9d0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 shutdown_connections 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2768078e50 0x7f276807b310 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f279815f900 0x7f27980c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f27981527f0 0x7f27980cd170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 --2- 10.20.193.6:0/2471548596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2798073530 0x7f27981f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 >> 10.20.193.6:0/2471548596 conn(0x7f27980ce730 msgr2=0x7f27980d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 shutdown_connections 2026-02-22T23:23:29.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 -- 10.20.193.6:0/2471548596 wait complete. 2026-02-22T23:23:29.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.651+0000 7f279e2ea640 1 librados: shutdown 2026-02-22T23:23:29.658 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-22T23:23:29.659 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-22T23:23:29.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.1 2026-02-22T23:23:29.662 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:29.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.670+0000 7fafeda90640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fafe815f650 0x7fafe80ceca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60344/0 (socket says 10.20.193.6:60344) 2026-02-22T23:23:29.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.670+0000 7fafeda90640 1 -- 10.20.193.6:0/1853172588 learned_addr learned my addr 10.20.193.6:0/1853172588 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.670+0000 7fafee291640 1 --2- 10.20.193.6:0/1853172588 >> v2:10.20.193.40:3300/0 conn(0x7fafe80d1ea0 0x7fafe80c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.670+0000 7fafee291640 10 monclient(hunting): get_auth_request con 0x7fafe80d1ea0 auth_method 0 2026-02-22T23:23:29.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.670+0000 7fafee291640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.670+0000 7fafee291640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.670+0000 7fafee291640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.671+0000 7fafee291640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.671+0000 7fafee291640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.671+0000 7fafee291640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 10 monclient(hunting): handle_auth_done global_id 4525 payload 274 2026-02-22T23:23:29.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 1 -- 10.20.193.6:0/1853172588 >> v2:10.20.193.80:3300/0 conn(0x7fafe815fa30 msgr2=0x7fafe80d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 1 --2- 10.20.193.6:0/1853172588 >> v2:10.20.193.80:3300/0 conn(0x7fafe815fa30 0x7fafe80d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 1 -- 10.20.193.6:0/1853172588 >> v1:10.20.193.40:6789/0 conn(0x7fafe815f650 legacy=0x7fafe80ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.672+0000 7fafee291640 1 -- 10.20.193.6:0/1853172588 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe80c9020 con 0x7fafe80d1ea0 2026-02-22T23:23:29.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.674+0000 7fafee291640 1 --2- 10.20.193.6:0/1853172588 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80c9b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fafe80ce0a0 tx=0x7fafe402f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea95d200f9c8fa0d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.675+0000 7f59bbfff640 1 -- 10.20.193.6:0/1300932326 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f59d001d030 con 0x7f59e01527f0 2026-02-22T23:23:29.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.675+0000 7f59bbfff640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f59b0078e50 0x7f59b007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.675+0000 7f59df7fe640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f59b0078e50 0x7f59b007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.675+0000 7f59df7fe640 10 monclient: get_auth_request con 0x7f59b0078e50 auth_method 0 2026-02-22T23:23:29.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.675+0000 7f59df7fe640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f59b0078e50 0x7f59b007b310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59cc007fd0 tx=0x7f59cc007ba0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.675+0000 7f59bbfff640 10 client.4580.objecter ms_handle_connect 0x7f59b0078e50 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 1 -- 10.20.193.6:0/1853172588 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe403c070 con 0x7fafe80d1ea0 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: dump: 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.678 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 1 -- 10.20.193.6:0/1853172588 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe402fe40 con 0x7fafe80d1ea0 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 1 -- 10.20.193.6:0/1853172588 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4037660 con 0x7fafe80d1ea0 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 5 monclient: authenticate success, global_id 4525 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 4 set_mon_vals no callback set 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafeca8e640 10 monclient: dump: 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.680 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7f59bbfff640 1 -- 10.20.193.6:0/1300932326 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f59d001de00 con 0x7f59e01527f0 2026-02-22T23:23:29.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7f59bbfff640 10 client.4580.objecter ms_dispatch2 0x7f59e0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7f59bbfff640 3 client.4580.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7f59bbfff640 3 client.4580.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7f59bbfff640 20 client.4580.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 10 monclient: shutdown 2026-02-22T23:23:29.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 -- 10.20.193.6:0/1853172588 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 msgr2=0x7fafe80c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1853172588 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80c9b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fafe80ce0a0 tx=0x7fafe402f130 comp rx=0 tx=0).stop 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 -- 10.20.193.6:0/1853172588 shutdown_connections 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1853172588 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80c9b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1853172588 >> v2:10.20.193.80:3300/0 conn(0x7fafe815fa30 0x7fafe80d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 -- 10.20.193.6:0/1853172588 >> 10.20.193.6:0/1853172588 conn(0x7fafe8072fc0 msgr2=0x7fafe80733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 -- 10.20.193.6:0/1853172588 shutdown_connections 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.676+0000 7fafefd1b640 1 -- 10.20.193.6:0/1853172588 wait complete. 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 10 monclient: monmap: 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.685 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 1 librados: starting msgr at 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 1 librados: starting objecter 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 1 Processor -- start 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 1 -- start start 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 1 librados: setting wanted keys 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 1 librados: calling monclient init 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.677+0000 7fafefd1b640 10 monclient: init 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): picked mon.noname-b con 0x7fafe80d1ea0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fafe815f650 0x7fafe81e9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): picked mon.noname-a con 0x7fafe815f650 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fafe815fa30 0x7fafe81ea230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): picked mon.noname-c con 0x7fafe815fa30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fafe8160150 con 0x7fafe815f650 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fafe80c95e0 con 0x7fafe80d1ea0 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fafe81602d0 con 0x7fafe815fa30 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafefd1b640 10 monclient(hunting): authenticate will time out at 1084.405096s 2026-02-22T23:23:29.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36636/0 (socket says 10.20.193.6:36636) 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 1 -- 10.20.193.6:0/1991861796 learned_addr learned my addr 10.20.193.6:0/1991861796 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 10 monclient(hunting): get_auth_request con 0x7fafe80d1ea0 auth_method 0 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.678+0000 7fafeda90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.679+0000 7fafeda90640 10 monclient(hunting): handle_auth_done global_id 4531 payload 824 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 1 -- 10.20.193.6:0/1991861796 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fafe815fa30 msgr2=0x7fafe81ea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fafe815fa30 0x7fafe81ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 1 -- 10.20.193.6:0/1991861796 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fafe815f650 msgr2=0x7fafe81e9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fafe815f650 0x7fafe81e9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 1 -- 10.20.193.6:0/1991861796 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe81e7550 con 0x7fafe80d1ea0 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafeda90640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80d0da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafd800c4d0 tx=0x7fafd800c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafd67fc640 10 client.?.objecter ms_handle_connect 0x7fafe80d1ea0 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.680+0000 7fafd67fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 1 -- 10.20.193.6:0/1991861796 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8017070 con 0x7fafe80d1ea0 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: dump: 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.689 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 5 monclient: authenticate success, global_id 4531 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 monclient: _renew_subs 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe81e7830 con 0x7fafe80d1ea0 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 client.4531.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 monclient: _renew_subs 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe81e7df0 con 0x7fafe80d1ea0 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 1 librados: init done 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe81e9950 con 0x7fafe80d1ea0 2026-02-22T23:23:29.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7f59bbfff640 1 -- 10.20.193.6:0/1300932326 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f59e00cc640 con 0x7f59e01527f0 2026-02-22T23:23:29.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7f59bbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7f59bbfff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 1 -- 10.20.193.6:0/1991861796 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd8005de0 con 0x7fafe80d1ea0 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 1 -- 10.20.193.6:0/1991861796 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8021060 con 0x7fafe80d1ea0 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd7fff640 4 set_mon_vals no callback set 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.682+0000 7fafd67fc640 10 monclient: dump: 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.692 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.687+0000 7fbc877fe640 1 -- 10.20.193.6:0/2129818887 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc9c061f20 con 0x7fbca806ff20 2026-02-22T23:23:29.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.687+0000 7fbc877fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:29.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.687+0000 7fbc877fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.693 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 10 librados: watch_flush enter 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 10 librados: watch_flush exit 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 20 client.4568.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 10 client.4568.objecter successfully canceled tick 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbc74078e00 msgr2=0x7fbc7407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbc74078e00 0x7fbc7407b2c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbc90002770 tx=0x7fbc90007ad0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 10 monclient: shutdown 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbca806ff20 msgr2=0x7fbca81f1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbca806ff20 0x7fbca81f1d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00b580 tx=0x7fbc9c00ba50 comp rx=0 tx=0).stop 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 shutdown_connections 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbc74078e00 0x7fbc7407b2c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbca815a5e0 0x7fbca8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbca80c99a0 0x7fbca81e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 --2- 10.20.193.6:0/2129818887 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbca806ff20 0x7fbca81f1d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 >> 10.20.193.6:0/2129818887 conn(0x7fbca80cd780 msgr2=0x7fbca814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 shutdown_connections 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 -- 10.20.193.6:0/2129818887 wait complete. 2026-02-22T23:23:29.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.690+0000 7fbcad36a640 1 librados: shutdown 2026-02-22T23:23:29.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.8 2026-02-22T23:23:29.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.718+0000 7f95c18f5640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.718+0000 7f95c18f5640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.718+0000 7f95c18f5640 10 monclient: monmap: 2026-02-22T23:23:29.719 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.719130+0000 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.719130+0000 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.720 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.720 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 1 Processor -- start 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 1 -- start start 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient: init 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient: _add_conns ranks=[4,1,2,3,0,5] 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f95bc1527f0 0x7f95bc073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient(hunting): picked mon.noname-b con 0x7f95bc1527f0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f95bc15d2c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f95bc15d730 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc073c20 con 0x7f95bc15d2c0 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95bc073f00 con 0x7f95bc15d730 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f95bc074b10 con 0x7f95bc1527f0 2026-02-22T23:23:29.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.719+0000 7f95c18f5640 10 monclient(hunting): authenticate will time out at 1084.446647s 2026-02-22T23:23:29.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.720+0000 7f95bb7fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f95bc15d730 0x7f95bc0ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36252/0 (socket says 10.20.193.6:36252) 2026-02-22T23:23:29.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.720+0000 7f95bb7fe640 1 -- 10.20.193.6:0/3206385846 learned_addr learned my addr 10.20.193.6:0/3206385846 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.721+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1724564437 0 0) 0x7f95bc073f00 con 0x7f95bc15d730 2026-02-22T23:23:29.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.721+0000 7f95b9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.721+0000 7f95b9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.721+0000 7f95b9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.721+0000 7f95b9ffb640 10 monclient(hunting): my global_id is 4583 2026-02-22T23:23:29.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.721+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95a8003620 con 0x7f95bc15d730 2026-02-22T23:23:29.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2088410370 0 0) 0x7f95a8003620 con 0x7f95bc15d730 2026-02-22T23:23:29.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 >> v2:10.20.193.40:3300/0 conn(0x7f95bc1527f0 msgr2=0x7f95bc073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 --2- 10.20.193.6:0/3206385846 >> v2:10.20.193.40:3300/0 conn(0x7f95bc1527f0 0x7f95bc073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 >> v1:10.20.193.40:6789/0 conn(0x7f95bc15d2c0 legacy=0x7f95bc15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc0741e0 con 0x7f95bc15d730 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f95ac003280 con 0x7f95bc15d730 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95c18f5640 5 monclient: authenticate success, global_id 4583 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95c18f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: dump: 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.727 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 >> v1:10.20.193.80:6789/0 conn(0x7f95bc15d730 legacy=0x7f95bc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8003ba0 0x7f95a8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): picked mon.a con 0x7f95a8003ba0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95a8004900 0x7f95a8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): picked mon.c con 0x7f95a8004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f95a8005510 0x7f95a800bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): picked mon.b con 0x7f95a8005510 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc074b10 con 0x7f95a8003ba0 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95a800c450 con 0x7f95a8005510 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f95a800c650 con 0x7f95a8004900 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95b9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95c18f5640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.723+0000 7f95baffd640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8003ba0 0x7f95a8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95baffd640 10 monclient(hunting): get_auth_request con 0x7f95a8003ba0 auth_method 0 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95baffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95baffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95baffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95ba7fc640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95a8004900 0x7f95a8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95ba7fc640 10 monclient(hunting): get_auth_request con 0x7f95a8004900 auth_method 0 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95ba7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95ba7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.724+0000 7f95ba7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.725+0000 7f95ba7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.725+0000 7f95ba7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.725+0000 7f95ba7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 10 monclient(hunting): handle_auth_done global_id 4583 payload 293 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 1 -- 10.20.193.6:0/3206385846 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f95a8005510 msgr2=0x7f95a800bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f95a8005510 0x7f95a800bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 1 -- 10.20.193.6:0/3206385846 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8003ba0 msgr2=0x7f95a8003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8003ba0 0x7f95a8003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 1 monclient: found mon.c 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 1 -- 10.20.193.6:0/3206385846 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a8003620 con 0x7f95a8004900 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.726+0000 7f95bb7fe640 1 -- 10.20.193.6:0/3206385846 reap_dead start 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f95ba7fc640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95a8004900 0x7f95a8004d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95a4002f60 tx=0x7f95a402eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a3151892effd81 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f508d18f640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f508d18f640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f508d18f640 10 monclient: monmap: 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.727993+0000 2026-02-22T23:23:29.732 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.727993+0000 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f2de73ca640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f2de73ca640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f2de73ca640 10 monclient: monmap: 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.728248+0000 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.728248+0000 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.734 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f508d18f640 1 Processor -- start 2026-02-22T23:23:29.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f508d18f640 1 -- start start 2026-02-22T23:23:29.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.727+0000 7f508d18f640 10 monclient: init 2026-02-22T23:23:29.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 1 Processor -- start 2026-02-22T23:23:29.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 1 -- start start 2026-02-22T23:23:29.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient: init 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient: _add_conns ranks=[2,3,0,4,1,5] 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f508806ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f5088155eb0 0x7f5088070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient(hunting): picked mon.noname-a con 0x7f5088155eb0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f508815c970 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50880725b0 con 0x7f508815c970 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5088072890 con 0x7f508806ff20 2026-02-22T23:23:29.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5088072b70 con 0x7f5088155eb0 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f508d18f640 10 monclient(hunting): authenticate will time out at 1084.455414s 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4002b30 con 0x7f95a8004900 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 10 monclient: dump: 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.737 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.738 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.738 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.738 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.738 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95c18f5640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.738 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient: _add_conns ranks=[5,0,4,1,2,3] 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f2de01527f0 0x7f2de0073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient(hunting): picked mon.noname-c con 0x7f2de01527f0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f2de015d2c0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f2de015d730 0x7f2de00ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient(hunting): picked mon.noname-b con 0x7f2de015d730 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de0073c20 con 0x7f2de015d2c0 2026-02-22T23:23:29.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f2de0074b10 con 0x7f2de015d730 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2de0073f00 con 0x7f2de01527f0 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f2de73ca640 10 monclient(hunting): authenticate will time out at 1084.455724s 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f5086575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f5088155eb0 0x7f5088070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a4005290 con 0x7f95a8004900 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 1 -- 10.20.193.6:0/3206385846 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4003710 con 0x7f95a8004900 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95c18f5640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95c18f5640 4 set_mon_vals no callback set 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.728+0000 7f95b9ffb640 10 monclient: dump: 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.741 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f5088155eb0 0x7f5088070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39950/0 (socket says 10.20.193.6:39950) 2026-02-22T23:23:29.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 1 -- 10.20.193.6:0/3462788211 learned_addr learned my addr 10.20.193.6:0/3462788211 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 10 monclient(hunting): get_auth_request con 0x7f5088155eb0 auth_method 0 2026-02-22T23:23:29.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f2de01527f0 0x7f2de0073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 10 monclient: shutdown 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 1 -- 10.20.193.6:0/3206385846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95a8004900 msgr2=0x7f95a8004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95a8004900 0x7f95a8004d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95a4002f60 tx=0x7f95a402eca0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 1 -- 10.20.193.6:0/3206385846 shutdown_connections 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3206385846 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95a8004900 0x7f95a8004d80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 1 -- 10.20.193.6:0/3206385846 >> 10.20.193.6:0/3206385846 conn(0x7f95bc0ce730 msgr2=0x7f95bc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 1 -- 10.20.193.6:0/3206385846 shutdown_connections 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 1 -- 10.20.193.6:0/3206385846 wait complete. 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f2de01527f0 0x7f2de0073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58744/0 (socket says 10.20.193.6:58744) 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 1 -- 10.20.193.6:0/1109523012 learned_addr learned my addr 10.20.193.6:0/1109523012 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 10 monclient(hunting): get_auth_request con 0x7f2de01527f0 auth_method 0 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f5086575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f2de513f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.729+0000 7f95c18f5640 10 monclient: monmap: 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.745 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 librados: starting msgr at 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 librados: starting objecter 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 Processor -- start 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 -- start start 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 librados: setting wanted keys 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 librados: calling monclient init 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient: init 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8004900 0x7f95bc0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): picked mon.noname-a con 0x7f95a8004900 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f95bc0c9d10 0x7f95bc0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): picked mon.noname-b con 0x7f95bc0c9d10 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95bc158c30 0x7f95bc1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): picked mon.noname-c con 0x7f95bc158c30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc074fa0 con 0x7f95a8004900 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc0747a0 con 0x7f95bc0c9d10 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc075120 con 0x7f95bc158c30 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f95c18f5640 10 monclient(hunting): authenticate will time out at 1084.457719s 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 10 monclient(hunting): handle_auth_done global_id 4586 payload 274 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 1 -- 10.20.193.6:0/1109523012 >> v2:10.20.193.40:3300/0 conn(0x7f2de015d730 msgr2=0x7f2de00ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 1 --2- 10.20.193.6:0/1109523012 >> v2:10.20.193.40:3300/0 conn(0x7f2de015d730 0x7f2de00ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 1 -- 10.20.193.6:0/1109523012 >> v1:10.20.193.6:6789/0 conn(0x7f2de015d2c0 legacy=0x7f2de015f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.730+0000 7f2de513f640 1 -- 10.20.193.6:0/1109523012 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de00741e0 con 0x7f2de01527f0 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 10 monclient(hunting): handle_auth_done global_id 4524 payload 274 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 1 -- 10.20.193.6:0/3462788211 >> v1:10.20.193.80:6789/0 conn(0x7f508806ff20 legacy=0x7f50881557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 1 -- 10.20.193.6:0/3462788211 >> v1:10.20.193.6:6789/0 conn(0x7f508815c970 legacy=0x7f50880c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f5086575640 1 -- 10.20.193.6:0/3462788211 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5088072e50 con 0x7f5088155eb0 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.731+0000 7f95bb7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95bc158c30 0x7f95bc1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f2de513f640 1 --2- 10.20.193.6:0/1109523012 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de01527f0 0x7f2de0073530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2de006e9a0 tx=0x7f2dd002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7393304aba44157a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95bc158c30 0x7f95bc1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58760/0 (socket says 10.20.193.6:58760) 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 1 -- 10.20.193.6:0/3586189641 learned_addr learned my addr 10.20.193.6:0/3586189641 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 10 monclient(hunting): get_auth_request con 0x7f95bc158c30 auth_method 0 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.732+0000 7f95bb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f5086575640 1 --2- 10.20.193.6:0/3462788211 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5088155eb0 0x7f5088070300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5070009970 tx=0x7f507002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc457469f6afe12 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f95baffd640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8004900 0x7f95bc0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f95baffd640 10 monclient(hunting): get_auth_request con 0x7f95a8004900 auth_method 0 2026-02-22T23:23:29.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f95baffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f95baffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f95baffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f7e92aa0640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:29.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f7e92aa0640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:29.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f7e92aa0640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.734+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7f7e8c1573f0 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95baffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95baffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95baffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 10 monclient(hunting): handle_auth_done global_id 4589 payload 824 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 1 -- 10.20.193.6:0/3586189641 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f95bc0c9d10 msgr2=0x7f95bc0ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f95bc0c9d10 0x7f95bc0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 1 -- 10.20.193.6:0/3586189641 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8004900 msgr2=0x7f95bc0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8004900 0x7f95bc0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.735+0000 7f95bb7fe640 1 -- 10.20.193.6:0/3586189641 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc0ca6d0 con 0x7f95bc158c30 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.736+0000 7f95bb7fe640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95bc158c30 0x7f95bc1590e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0ce270 tx=0x7f95ac00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.736+0000 7f9593fff640 10 client.?.objecter ms_handle_connect 0x7f95bc158c30 2026-02-22T23:23:29.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.736+0000 7f9593fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 1 -- 10.20.193.6:0/1109523012 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd003c070 con 0x7f2de01527f0 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: dump: 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.755 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 1 -- 10.20.193.6:0/1109523012 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd002fb40 con 0x7f2de01527f0 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 1 -- 10.20.193.6:0/1109523012 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0037440 con 0x7f2de01527f0 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 5 monclient: authenticate success, global_id 4586 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 4 set_mon_vals no callback set 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2dcffff640 10 monclient: dump: 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.756 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 10 monclient: shutdown 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 -- 10.20.193.6:0/1109523012 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de01527f0 msgr2=0x7f2de0073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 --2- 10.20.193.6:0/1109523012 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de01527f0 0x7f2de0073530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2de006e9a0 tx=0x7f2dd002eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 -- 10.20.193.6:0/1109523012 shutdown_connections 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 --2- 10.20.193.6:0/1109523012 >> v2:10.20.193.40:3300/0 conn(0x7f2de015d730 0x7f2de00ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 --2- 10.20.193.6:0/1109523012 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de01527f0 0x7f2de0073530 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 -- 10.20.193.6:0/1109523012 >> 10.20.193.6:0/1109523012 conn(0x7f2de00ce730 msgr2=0x7f2de00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 -- 10.20.193.6:0/1109523012 shutdown_connections 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.737+0000 7f2de73ca640 1 -- 10.20.193.6:0/1109523012 wait complete. 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient: monmap: 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.757 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 librados: starting msgr at 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 librados: starting objecter 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 Processor -- start 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 -- start start 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 librados: setting wanted keys 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 librados: calling monclient init 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient: init 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2de01527f0 0x7f2de01f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): picked mon.noname-a con 0x7f2de01527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2de015d2c0 0x7f2de00ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): picked mon.noname-b con 0x7f2de015d2c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de015d730 0x7f2de00c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): picked mon.noname-c con 0x7f2de015d730 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0074fa0 con 0x7f2de01527f0 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2de00747a0 con 0x7f2de015d2c0 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0075120 con 0x7f2de015d730 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.738+0000 7f2de73ca640 10 monclient(hunting): authenticate will time out at 1084.465924s 2026-02-22T23:23:29.760 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:23:29.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 1 -- 10.20.193.6:0/3586189641 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00acf0 con 0x7f95bc158c30 2026-02-22T23:23:29.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: dump: 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 1 -- 10.20.193.6:0/3586189641 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ac00ae90 con 0x7f95bc158c30 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 1 -- 10.20.193.6:0/3586189641 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac014660 con 0x7f95bc158c30 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 5 monclient: authenticate success, global_id 4589 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 10 monclient: _renew_subs 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95b97fa640 4 set_mon_vals no callback set 2026-02-22T23:23:29.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc1f42b0 con 0x7f95bc158c30 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f9593fff640 10 monclient: dump: 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 10 client.4589.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 10 monclient: _renew_subs 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc0c6ce0 con 0x7f95bc158c30 2026-02-22T23:23:29.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f95c18f5640 1 librados: init done 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de513f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2de01527f0 0x7f2de01f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de5940640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de015d730 0x7f2de00c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de513f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2de01527f0 0x7f2de01f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39968/0 (socket says 10.20.193.6:39968) 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de513f640 1 -- 10.20.193.6:0/449808492 learned_addr learned my addr 10.20.193.6:0/449808492 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de513f640 10 monclient(hunting): get_auth_request con 0x7f2de01527f0 auth_method 0 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de513f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de513f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.739+0000 7f2de513f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f95c18f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f95c18f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f95c18f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc152bd0 con 0x7f95bc158c30 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f2de513f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f2de513f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f2de513f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f2de5940640 10 monclient(hunting): get_auth_request con 0x7f2de015d730 auth_method 0 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f2de5940640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f2de5940640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.740+0000 7f2de5940640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 10 monclient(hunting): handle_auth_done global_id 4530 payload 824 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 1 -- 10.20.193.6:0/449808492 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de015d730 msgr2=0x7f2de00c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de015d730 0x7f2de00c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 1 -- 10.20.193.6:0/449808492 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2de015d2c0 msgr2=0x7f2de00ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2de015d2c0 0x7f2de00ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.741+0000 7f2de513f640 1 -- 10.20.193.6:0/449808492 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de00c7550 con 0x7f2de01527f0 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.743+0000 7f2de513f640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2de01527f0 0x7f2de01f75c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009a70 tx=0x7f2dd00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.743+0000 7f2dcdffb640 10 client.?.objecter ms_handle_connect 0x7f2de01527f0 2026-02-22T23:23:29.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.743+0000 7f2dcdffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.775+0000 7f59e5ac4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:29.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.775+0000 7f59e5ac4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:29.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.775+0000 7f59e5ac4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.775+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f59e00747a0 con 0x7f59e01527f0 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.787+0000 7f320cd33640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.787+0000 7f320cd33640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.787+0000 7f320cd33640 10 monclient: monmap: 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.788593+0000 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.788593+0000 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 1 Processor -- start 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 1 -- start start 2026-02-22T23:23:29.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient: init 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient: _add_conns ranks=[5,1,2,0,3,4] 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f32081527f0 0x7f3208073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient(hunting): picked mon.noname-c con 0x7f32081527f0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f320815d2c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f320815d730 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3208073c20 con 0x7f320815d2c0 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3208073f00 con 0x7f320815d730 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3208074b10 con 0x7f32081527f0 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.788+0000 7f320cd33640 10 monclient(hunting): authenticate will time out at 1084.515960s 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.789+0000 7fafd67fc640 1 -- 10.20.193.6:0/1991861796 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fafd8007270 con 0x7fafe80d1ea0 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.789+0000 7fafd67fc640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafb8078e30 0x7fafb807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.789+0000 7fafed28f640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafb8078e30 0x7fafb807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.789+0000 7fafed28f640 10 monclient: get_auth_request con 0x7fafb8078e30 auth_method 0 2026-02-22T23:23:29.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.790+0000 7fafd67fc640 10 client.4531.objecter ms_handle_connect 0x7fafb8078e30 2026-02-22T23:23:29.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.790+0000 7fafed28f640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafb8078e30 0x7fafb807b2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fafe80cfc50 tx=0x7fafdc006cf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.791+0000 7fafd67fc640 1 -- 10.20.193.6:0/1991861796 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fafd80972c0 con 0x7fafe80d1ea0 2026-02-22T23:23:29.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.791+0000 7fafd67fc640 10 client.4531.objecter ms_dispatch2 0x7fafe8000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.791+0000 7fafd67fc640 3 client.4531.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.791+0000 7fafd67fc640 3 client.4531.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.791+0000 7fafd67fc640 20 client.4531.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.797+0000 7fafd67fc640 1 -- 10.20.193.6:0/1991861796 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fafd8061a90 con 0x7fafe80d1ea0 2026-02-22T23:23:29.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.797+0000 7fafd67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.797+0000 7fafd67fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.805+0000 7f3205d74640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f320815d2c0 0x7f320815f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60360/0 (socket says 10.20.193.6:60360) 2026-02-22T23:23:29.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.805+0000 7f3205d74640 1 -- 10.20.193.6:0/2553093847 learned_addr learned my addr 10.20.193.6:0/2553093847 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.805+0000 7f7e817fa640 1 -- 10.20.193.6:0/2712884179 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f7e88062dd0 con 0x7f7e8c15a5e0 2026-02-22T23:23:29.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.805+0000 7f7e817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:29.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.805+0000 7f7e817fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.807 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:29.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435137601 0 0) 0x7f3208073c20 con 0x7f320815d2c0 2026-02-22T23:23:29.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f3205573640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f3205573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f3205573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f3205573640 10 monclient(hunting): my global_id is 4534 2026-02-22T23:23:29.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31f4003620 con 0x7f320815d2c0 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f7e92aa0640 10 librados: watch_flush enter 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f7e92aa0640 10 librados: watch_flush exit 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.807+0000 7f7e92aa0640 20 client.4522.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 10 client.4522.objecter successfully canceled tick 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7e5c078e00 msgr2=0x7f7e5c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7e5c078e00 0x7f7e5c07b2c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009970 tx=0x7f7e7c031000 comp rx=0 tx=0).stop 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 10 monclient: shutdown 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c15a5e0 msgr2=0x7f7e8c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c15a5e0 0x7f7e8c06e4a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800cba0 tx=0x7f7e8800a430 comp rx=0 tx=0).stop 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 shutdown_connections 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7e5c078e00 0x7f7e5c07b2c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7e8c15a5e0 0x7f7e8c06e4a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7e8c0c99a0 0x7f7e8c1f64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 --2- 10.20.193.6:0/2712884179 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7e8c06ff20 0x7f7e8c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 >> 10.20.193.6:0/2712884179 conn(0x7f7e8c0cd780 msgr2=0x7f7e8c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 shutdown_connections 2026-02-22T23:23:29.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 -- 10.20.193.6:0/2712884179 wait complete. 2026-02-22T23:23:29.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.808+0000 7f7e92aa0640 1 librados: shutdown 2026-02-22T23:23:29.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 215651490 0 0) 0x7f31f4003620 con 0x7f320815d2c0 2026-02-22T23:23:29.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 >> v2:10.20.193.80:3300/0 conn(0x7f32081527f0 msgr2=0x7f3208073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 --2- 10.20.193.6:0/2553093847 >> v2:10.20.193.80:3300/0 conn(0x7f32081527f0 0x7f3208073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 >> v1:10.20.193.80:6789/0 conn(0x7f320815d730 legacy=0x7f32080ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32080741e0 con 0x7f320815d2c0 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f31f0002f00 con 0x7f320815d2c0 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f320cd33640 5 monclient: authenticate success, global_id 4534 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f320cd33640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:23:29.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: dump: 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 >> v1:10.20.193.40:6789/0 conn(0x7f320815d2c0 legacy=0x7f320815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:29.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f31f4003ba0 0x7f31f4003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): picked mon.b con 0x7f31f4003ba0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f31f4004900 0x7f31f4004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): picked mon.c con 0x7f31f4004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f31f4005510 0x7f31f400bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): picked mon.a con 0x7f31f4005510 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3208074b10 con 0x7f31f4005510 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f31f400c450 con 0x7f31f4003ba0 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f31f400c650 con 0x7f31f4004900 2026-02-22T23:23:29.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3205573640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f320cd33640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.809+0000 7f3206d76640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f31f4005510 0x7f31f400bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.810+0000 7f3206d76640 10 monclient(hunting): get_auth_request con 0x7f31f4005510 auth_method 0 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.810+0000 7f3206d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.810+0000 7f3206d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.810+0000 7f3206d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.810+0000 7f3206d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.810+0000 7f3206d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.810+0000 7f3206d76640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f31f4003ba0 0x7f31f4003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 10 monclient(hunting): get_auth_request con 0x7f31f4003ba0 auth_method 0 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.811+0000 7f3206575640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 10 monclient(hunting): handle_auth_done global_id 4534 payload 293 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 1 -- 10.20.193.6:0/2553093847 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f31f4004900 msgr2=0x7f31f4004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f31f4004900 0x7f31f4004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 1 -- 10.20.193.6:0/2553093847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f31f4005510 msgr2=0x7f31f400bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f31f4005510 0x7f31f400bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 1 monclient: found mon.b 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 1 -- 10.20.193.6:0/2553093847 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f4003620 con 0x7f31f4003ba0 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206d76640 1 -- 10.20.193.6:0/2553093847 reap_dead start 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.812+0000 7f3206575640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.813+0000 7f3206575640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f31f4003ba0 0x7f31f4003fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009980 tx=0x7f31fc02ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b164cba9db6f107 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.814+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc03c070 con 0x7f31f4003ba0 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.814+0000 7f3205573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.814+0000 7f3205573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.814+0000 7f3205573640 10 monclient: dump: 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.814+0000 7f320cd33640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31fc02f770 con 0x7f31f4003ba0 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f3205573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f3205573640 1 -- 10.20.193.6:0/2553093847 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc009550 con 0x7f31f4003ba0 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f3205573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 4 set_mon_vals no callback set 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f3205573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f3205573640 10 monclient: dump: 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 10 monclient: shutdown 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 -- 10.20.193.6:0/2553093847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f31f4003ba0 msgr2=0x7f31f4003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f31f4003ba0 0x7f31f4003fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009980 tx=0x7f31fc02ed10 comp rx=0 tx=0).stop 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 -- 10.20.193.6:0/2553093847 shutdown_connections 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 --2- 10.20.193.6:0/2553093847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f31f4003ba0 0x7f31f4003fa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 -- 10.20.193.6:0/2553093847 >> 10.20.193.6:0/2553093847 conn(0x7f32080ce730 msgr2=0x7f32080ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 -- 10.20.193.6:0/2553093847 shutdown_connections 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 -- 10.20.193.6:0/2553093847 wait complete. 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 10 monclient: monmap: 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.821 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 librados: starting msgr at 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 librados: starting objecter 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.815+0000 7f320cd33640 1 Processor -- start 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 -- start start 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 librados: setting wanted keys 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 librados: calling monclient init 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient: init 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f31f4003ba0 0x7f3208156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): picked mon.noname-c con 0x7f31f4003ba0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32080c9e50 0x7f32081f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): picked mon.noname-a con 0x7f32080c9e50 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32080ca2d0 0x7f32081f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): picked mon.noname-b con 0x7f32080ca2d0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3208074fa0 con 0x7f32080c9e50 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f32080747a0 con 0x7f32080ca2d0 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3208075120 con 0x7f31f4003ba0 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f320cd33640 10 monclient(hunting): authenticate will time out at 1084.543414s 2026-02-22T23:23:29.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3206d76640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32080ca2d0 0x7f32081f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3205d74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32080c9e50 0x7f32081f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.13 2026-02-22T23:23:29.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3206d76640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32080ca2d0 0x7f32081f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36696/0 (socket says 10.20.193.6:36696) 2026-02-22T23:23:29.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3206d76640 1 -- 10.20.193.6:0/1170235400 learned_addr learned my addr 10.20.193.6:0/1170235400 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3206d76640 10 monclient(hunting): get_auth_request con 0x7f32080ca2d0 auth_method 0 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3206d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3206d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.816+0000 7f3206d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3205d74640 10 monclient(hunting): get_auth_request con 0x7f32080c9e50 auth_method 0 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3205d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3205d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3205d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3206d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3206d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3206d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3205d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3205d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f3205d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f6b6ffeb640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f6b6ffeb640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f6b6ffeb640 10 monclient: monmap: 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.818137+0000 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.818137+0000 2026-02-22T23:23:29.824 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.817+0000 7f6b6ffeb640 1 Processor -- start 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 1 -- start start 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient: init 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 10 monclient(hunting): handle_auth_done global_id 4537 payload 824 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 1 -- 10.20.193.6:0/1170235400 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f31f4003ba0 msgr2=0x7f3208156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f31f4003ba0 0x7f3208156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 1 -- 10.20.193.6:0/1170235400 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32080c9e50 msgr2=0x7f32081f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32080c9e50 0x7f32081f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f3206d76640 1 -- 10.20.193.6:0/1170235400 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3208157e40 con 0x7f32080ca2d0 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient: _add_conns ranks=[1,2,5,4,0,3] 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6b6815f650 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6b6815fa30 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6b680c9b00 0x7f6b680c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient(hunting): picked mon.noname-c con 0x7f6b680c9b00 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b680ca4b0 con 0x7f6b6815f650 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b680d2450 con 0x7f6b6815fa30 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6b680c8dd0 con 0x7f6b680c9b00 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.818+0000 7f6b6ffeb640 10 monclient(hunting): authenticate will time out at 1084.545604s 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.819+0000 7f6b6dd60640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f6b6815f650 0x7f6b680ceca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60374/0 (socket says 10.20.193.6:60374) 2026-02-22T23:23:29.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.819+0000 7f6b6dd60640 1 -- 10.20.193.6:0/3713515791 learned_addr learned my addr 10.20.193.6:0/3713515791 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.819+0000 7f3206d76640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32080ca2d0 0x7f32081f5830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f32080c88f0 tx=0x7f31f8004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.819+0000 7f31eeffd640 10 client.?.objecter ms_handle_connect 0x7f32080ca2d0 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.819+0000 7f31eeffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 1 -- 10.20.193.6:0/1170235400 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f8017020 con 0x7f32080ca2d0 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: dump: 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.827 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 1 -- 10.20.193.6:0/1170235400 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f800fa30 con 0x7f32080ca2d0 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 5 monclient: authenticate success, global_id 4537 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 monclient: _renew_subs 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3208158120 con 0x7f32080ca2d0 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 1 -- 10.20.193.6:0/1170235400 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f800fe40 con 0x7f32080ca2d0 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f3204d72640 4 set_mon_vals no callback set 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f31eeffd640 10 monclient: dump: 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 client.4537.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 monclient: _renew_subs 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32081f4af0 con 0x7f32080ca2d0 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 1 librados: init done 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.821+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3208152bd0 con 0x7f32080ca2d0 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.822+0000 7f6b6cd5e640 1 -- 10.20.193.6:0/3713515791 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560360017 0 0) 0x7f6b680ca4b0 con 0x7f6b6815f650 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.822+0000 7f6b6cd5e640 10 monclient(hunting): _negotiate 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.822+0000 7f6b6cd5e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.822+0000 7f6b6cd5e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.822+0000 7f6b6cd5e640 10 monclient(hunting): my global_id is 4540 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.822+0000 7f6b6cd5e640 1 -- 10.20.193.6:0/3713515791 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b50003620 con 0x7f6b6815f650 2026-02-22T23:23:29.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.828+0000 7fc456c8f640 10 monclient: get_monmap_and_config 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.828+0000 7fc456c8f640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.828+0000 7fc456c8f640 10 monclient: monmap: 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:29.829465+0000 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:29.829465+0000 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:29.830 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 1 Processor -- start 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 1 -- start start 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient: init 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient: _add_conns ranks=[3,4,1,0,2,5] 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc45006ff20 0x7fc450070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient(hunting): picked mon.noname-a con 0x7fc45006ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc4500c99a0 0x7fc450159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient(hunting): picked mon.noname-b con 0x7fc4500c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc45015a410 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc45015a860 con 0x7fc45015a410 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc450159a40 con 0x7fc45006ff20 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc4500c7e90 con 0x7fc4500c99a0 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.829+0000 7fc456c8f640 10 monclient(hunting): authenticate will time out at 1084.556805s 2026-02-22T23:23:29.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc45006ff20 0x7fc450070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc45006ff20 0x7fc450070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39986/0 (socket says 10.20.193.6:39986) 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 1 -- 10.20.193.6:0/1307398431 learned_addr learned my addr 10.20.193.6:0/1307398431 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 10 monclient(hunting): get_auth_request con 0x7fc45006ff20 auth_method 0 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.830+0000 7fc454a04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 10 monclient(hunting): handle_auth_done global_id 4536 payload 274 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 1 -- 10.20.193.6:0/1307398431 >> v2:10.20.193.40:3300/0 conn(0x7fc4500c99a0 msgr2=0x7fc450159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 1 --2- 10.20.193.6:0/1307398431 >> v2:10.20.193.40:3300/0 conn(0x7fc4500c99a0 0x7fc450159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 1 -- 10.20.193.6:0/1307398431 >> v1:10.20.193.40:6789/0 conn(0x7fc45015a410 legacy=0x7fc45014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.831+0000 7fc454a04640 1 -- 10.20.193.6:0/1307398431 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4500c8170 con 0x7fc45006ff20 2026-02-22T23:23:29.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.832+0000 7fc454a04640 1 --2- 10.20.193.6:0/1307398431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc45006ff20 0x7fc450070300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc440009970 tx=0x7fc44002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944accde247735d3 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.842+0000 7f9593fff640 1 -- 10.20.193.6:0/3586189641 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f95ac0133c0 con 0x7f95bc158c30 2026-02-22T23:23:29.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.842+0000 7f9593fff640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f958c078e50 0x7f958c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.842+0000 7f95baffd640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f958c078e50 0x7f958c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.842+0000 7f95baffd640 10 monclient: get_auth_request con 0x7f958c078e50 auth_method 0 2026-02-22T23:23:29.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.843+0000 7f9593fff640 10 client.4589.objecter ms_handle_connect 0x7f958c078e50 2026-02-22T23:23:29.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.843+0000 7f95baffd640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f958c078e50 0x7f958c07b310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95b0004800 tx=0x7f95b0009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.844+0000 7f9593fff640 1 -- 10.20.193.6:0/3586189641 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f95ac05ddb0 con 0x7f95bc158c30 2026-02-22T23:23:29.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.844+0000 7f9593fff640 10 client.4589.objecter ms_dispatch2 0x7f95bc000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.844+0000 7f9593fff640 3 client.4589.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.844+0000 7f9593fff640 3 client.4589.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.844+0000 7f9593fff640 20 client.4589.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.851+0000 7f9593fff640 1 -- 10.20.193.6:0/3586189641 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f95ac01a060 con 0x7f95bc158c30 2026-02-22T23:23:29.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.851+0000 7f9593fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.851+0000 7f9593fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.854+0000 7f59bbfff640 1 -- 10.20.193.6:0/1300932326 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f59d0024080 con 0x7f59e01527f0 2026-02-22T23:23:29.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.854+0000 7f59bbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:29.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.854+0000 7f59bbfff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.855 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 10 librados: watch_flush enter 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 10 librados: watch_flush exit 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 20 client.4580.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 10 client.4580.objecter successfully canceled tick 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f59b0078e50 msgr2=0x7f59b007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f59b0078e50 0x7f59b007b310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59cc007fd0 tx=0x7f59cc007ba0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 10 monclient: shutdown 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e01527f0 msgr2=0x7f59e01f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e01527f0 0x7f59e01f4950 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f59d000b780 tx=0x7f59d000bc50 comp rx=0 tx=0).stop 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 shutdown_connections 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f59b0078e50 0x7f59b007b310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f59e015f900 0x7f59e00c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f59e01527f0 0x7f59e01f4950 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 --2- 10.20.193.6:0/1300932326 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f59e0073530 0x7f59e00c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.856+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 >> 10.20.193.6:0/1300932326 conn(0x7f59e00ce730 msgr2=0x7f59e00cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.857+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 shutdown_connections 2026-02-22T23:23:29.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.857+0000 7f59e5ac4640 1 -- 10.20.193.6:0/1300932326 wait complete. 2026-02-22T23:23:29.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.857+0000 7f59e5ac4640 1 librados: shutdown 2026-02-22T23:23:29.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.862+0000 7f6b6e561640 1 --2- 10.20.193.6:0/3713515791 >> v2:10.20.193.80:3300/0 conn(0x7f6b680c9b00 0x7f6b680c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.863+0000 7f6b6e561640 10 monclient(hunting): get_auth_request con 0x7f6b680c9b00 auth_method 0 2026-02-22T23:23:29.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.863+0000 7f6b6e561640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.863+0000 7f6b6e561640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.863+0000 7f6b6e561640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.864+0000 7f6b6e561640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.864+0000 7f6b6e561640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.864+0000 7f6b6e561640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.4 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 10 monclient(hunting): handle_auth_done global_id 4595 payload 274 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 1 -- 10.20.193.6:0/3713515791 >> v1:10.20.193.80:6789/0 conn(0x7f6b6815fa30 legacy=0x7f6b680d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 1 -- 10.20.193.6:0/3713515791 >> v1:10.20.193.40:6789/0 conn(0x7f6b6815f650 legacy=0x7f6b680ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.867+0000 7f6b6e561640 1 -- 10.20.193.6:0/3713515791 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b680d2730 con 0x7f6b680c9b00 2026-02-22T23:23:29.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.870+0000 7f6b6e561640 1 --2- 10.20.193.6:0/3713515791 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b680c9b00 0x7f6b680c9f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009d90 tx=0x7f6b6402f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f386905e44eb8 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6cd5e640 1 -- 10.20.193.6:0/3713515791 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6403c070 con 0x7f6b680c9b00 2026-02-22T23:23:29.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6cd5e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6cd5e640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6cd5e640 10 monclient: dump: 2026-02-22T23:23:29.874 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.874 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.874 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6cd5e640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6cd5e640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6cd5e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6ffeb640 5 monclient: authenticate success, global_id 4595 2026-02-22T23:23:29.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.872+0000 7f6b6ffeb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6cd5e640 1 -- 10.20.193.6:0/3713515791 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b64037440 con 0x7f6b680c9b00 2026-02-22T23:23:29.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6cd5e640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6cd5e640 1 -- 10.20.193.6:0/3713515791 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640377f0 con 0x7f6b680c9b00 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6cd5e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6cd5e640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6cd5e640 10 monclient: dump: 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6ffeb640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.873+0000 7f6b6ffeb640 4 set_mon_vals no callback set 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 10 monclient: shutdown 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3713515791 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b680c9b00 msgr2=0x7f6b680c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3713515791 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b680c9b00 0x7f6b680c9f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009d90 tx=0x7f6b6402f1e0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3713515791 shutdown_connections 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3713515791 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b680c9b00 0x7f6b680c9f70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3713515791 >> 10.20.193.6:0/3713515791 conn(0x7f6b68072fc0 msgr2=0x7f6b680733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3713515791 shutdown_connections 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3713515791 wait complete. 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.874+0000 7f6b6ffeb640 10 monclient: monmap: 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.879 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 librados: starting msgr at 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 librados: starting objecter 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 Processor -- start 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 -- start start 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 librados: setting wanted keys 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 librados: calling monclient init 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient: init 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b680c9b00 0x7f6b68149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): picked mon.noname-a con 0x7f6b680c9b00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b6815f650 0x7f6b681f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): picked mon.noname-c con 0x7f6b6815f650 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b6815fa30 0x7f6b6814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): picked mon.noname-b con 0x7f6b6815fa30 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6b680c95e0 con 0x7f6b680c9b00 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6b680c9300 con 0x7f6b6815fa30 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6b680d2070 con 0x7f6b6815f650 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.875+0000 7f6b6ffeb640 10 monclient(hunting): authenticate will time out at 1084.602559s 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b6815f650 0x7f6b681f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b6815f650 0x7f6b681f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58804/0 (socket says 10.20.193.6:58804) 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 1 -- 10.20.193.6:0/3056384900 learned_addr learned my addr 10.20.193.6:0/3056384900 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 10 monclient(hunting): get_auth_request con 0x7f6b6815f650 auth_method 0 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.877+0000 7f6b6d55f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 10 monclient(hunting): handle_auth_done global_id 4601 payload 824 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 1 -- 10.20.193.6:0/3056384900 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b6815fa30 msgr2=0x7f6b6814be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b6815fa30 0x7f6b6814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 1 -- 10.20.193.6:0/3056384900 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b680c9b00 msgr2=0x7f6b68149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b680c9b00 0x7f6b68149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 1 monclient: found mon.noname-c 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.879+0000 7f6b6d55f640 1 -- 10.20.193.6:0/3056384900 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b681f47f0 con 0x7f6b6815f650 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.880+0000 7f6b6d55f640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b6815f650 0x7f6b681f3e20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c730 tx=0x7f6b5c00cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.880+0000 7f6b567fc640 10 client.?.objecter ms_handle_connect 0x7f6b6815f650 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.880+0000 7f6b567fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 1 -- 10.20.193.6:0/3056384900 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0052d0 con 0x7f6b6815f650 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: dump: 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 1 -- 10.20.193.6:0/3056384900 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5c005470 con 0x7f6b6815f650 2026-02-22T23:23:29.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 1 -- 10.20.193.6:0/3056384900 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0038d0 con 0x7f6b6815f650 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b57fff640 4 set_mon_vals no callback set 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b567fc640 10 monclient: dump: 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 5 monclient: authenticate success, global_id 4601 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 10 monclient: _renew_subs 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b681f4ad0 con 0x7f6b6815f650 2026-02-22T23:23:29.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 10 client.4601.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 10 monclient: _renew_subs 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b68149fd0 con 0x7f6b6815f650 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.881+0000 7f6b6ffeb640 1 librados: init done 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f6b6ffeb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f6b6ffeb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f6b6ffeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b680ca7a0 con 0x7f6b6815f650 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23cdffb640 1 -- 10.20.193.6:0/3630709713 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c403c070 con 0x7f23d006ff20 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23cdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23cdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23cdffb640 10 monclient: dump: 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23cdffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23cdffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23cdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23d53e4640 5 monclient: authenticate success, global_id 4521 2026-02-22T23:23:29.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.882+0000 7f23d53e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.883+0000 7fafefd1b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:29.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.883+0000 7fafefd1b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:29.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.883+0000 7fafefd1b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.883+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7fafe80d1960 con 0x7fafe80d1ea0 2026-02-22T23:23:29.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23cdffb640 1 -- 10.20.193.6:0/3630709713 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c402fb80 con 0x7f23d006ff20 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23cdffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23cdffb640 1 -- 10.20.193.6:0/3630709713 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4037440 con 0x7f23d006ff20 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23cdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23cdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23cdffb640 10 monclient: dump: 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23d53e4640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.887+0000 7f23d53e4640 4 set_mon_vals no callback set 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 10 monclient: shutdown 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 -- 10.20.193.6:0/3630709713 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d006ff20 msgr2=0x7f23d0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 --2- 10.20.193.6:0/3630709713 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d006ff20 0x7f23d0070300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f23c40099a0 tx=0x7f23c402ef50 comp rx=0 tx=0).stop 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 -- 10.20.193.6:0/3630709713 shutdown_connections 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 --2- 10.20.193.6:0/3630709713 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d006ff20 0x7f23d0070300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 -- 10.20.193.6:0/3630709713 >> 10.20.193.6:0/3630709713 conn(0x7f23d00cd780 msgr2=0x7f23d00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 -- 10.20.193.6:0/3630709713 shutdown_connections 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 -- 10.20.193.6:0/3630709713 wait complete. 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 10 monclient: monmap: 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 librados: starting msgr at 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.888+0000 7f23d53e4640 1 librados: starting objecter 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 Processor -- start 2026-02-22T23:23:29.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 -- start start 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 librados: setting wanted keys 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 librados: calling monclient init 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient: init 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f23d006ff20 0x7f23d01f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): picked mon.noname-c con 0x7f23d006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d00c99a0 0x7f23d006e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): picked mon.noname-a con 0x7f23d00c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f23d015a5e0 0x7f23d01f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): picked mon.noname-b con 0x7f23d015a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f23d0150520 con 0x7f23d00c99a0 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f23d00c8a10 con 0x7f23d015a5e0 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f23d01506a0 con 0x7f23d006ff20 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.889+0000 7f23d53e4640 10 monclient(hunting): authenticate will time out at 1084.616485s 2026-02-22T23:23:29.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 1 -- 10.20.193.6:0/1307398431 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44003c070 con 0x7fc45006ff20 2026-02-22T23:23:29.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:29.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: dump: 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 1 -- 10.20.193.6:0/1307398431 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44002fb20 con 0x7fc45006ff20 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 5 monclient: authenticate success, global_id 4536 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 1 -- 10.20.193.6:0/1307398431 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440037440 con 0x7fc45006ff20 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 4 set_mon_vals no callback set 2026-02-22T23:23:29.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc4477fe640 10 monclient: dump: 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 10 monclient: shutdown 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 -- 10.20.193.6:0/1307398431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc45006ff20 msgr2=0x7fc450070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 --2- 10.20.193.6:0/1307398431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc45006ff20 0x7fc450070300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc440009970 tx=0x7fc44002eef0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 -- 10.20.193.6:0/1307398431 shutdown_connections 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 --2- 10.20.193.6:0/1307398431 >> v2:10.20.193.40:3300/0 conn(0x7fc4500c99a0 0x7fc450159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 --2- 10.20.193.6:0/1307398431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc45006ff20 0x7fc450070300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 -- 10.20.193.6:0/1307398431 >> 10.20.193.6:0/1307398431 conn(0x7fc4500cd780 msgr2=0x7fc4500cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 -- 10.20.193.6:0/1307398431 shutdown_connections 2026-02-22T23:23:29.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 1 -- 10.20.193.6:0/1307398431 wait complete. 2026-02-22T23:23:29.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.896+0000 7fc456c8f640 10 monclient: monmap: 2026-02-22T23:23:29.900 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.900 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.900 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 librados: starting msgr at 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 librados: starting objecter 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 Processor -- start 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 -- start start 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 librados: setting wanted keys 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 librados: calling monclient init 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient: init 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc45006ff20 0x7fc4501f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): picked mon.noname-c con 0x7fc45006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4500c99a0 0x7fc4501f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): picked mon.noname-a con 0x7fc4500c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc45015a410 0x7fc4501f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): picked mon.noname-b con 0x7fc45015a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc4500c8a10 con 0x7fc4500c99a0 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4500c8730 con 0x7fc45015a410 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc450150520 con 0x7fc45006ff20 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.897+0000 7fc456c8f640 10 monclient(hunting): authenticate will time out at 1084.624784s 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 1 -- 10.20.193.6:0/3462788211 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507003c070 con 0x7f5088155eb0 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: dump: 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.902 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 1 -- 10.20.193.6:0/3462788211 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507002fbb0 con 0x7f5088155eb0 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 1 -- 10.20.193.6:0/3462788211 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070037440 con 0x7f5088155eb0 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f508d18f640 5 monclient: authenticate success, global_id 4524 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f508d18f640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f508d18f640 4 set_mon_vals no callback set 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.898+0000 7f5085d74640 10 monclient: dump: 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.903 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f3925ffb640 1 -- 10.20.193.6:0/1311347986 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3934086d60 con 0x7f393815ed40 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f3925ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f3925ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.904 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:29.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 10 monclient: shutdown 2026-02-22T23:23:29.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 1 -- 10.20.193.6:0/3462788211 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5088155eb0 msgr2=0x7f5088070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 1 --2- 10.20.193.6:0/3462788211 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5088155eb0 0x7f5088070300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5070009970 tx=0x7f507002eef0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 1 -- 10.20.193.6:0/3462788211 shutdown_connections 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 1 --2- 10.20.193.6:0/3462788211 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5088155eb0 0x7f5088070300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 1 -- 10.20.193.6:0/3462788211 >> 10.20.193.6:0/3462788211 conn(0x7f50880cd780 msgr2=0x7f50880cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 1 -- 10.20.193.6:0/3462788211 shutdown_connections 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 1 -- 10.20.193.6:0/3462788211 wait complete. 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 10 monclient: build_initial_monmap 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.899+0000 7f508d18f640 10 monclient: monmap: 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 librados: starting msgr at 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 librados: starting objecter 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 Processor -- start 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 -- start start 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 librados: setting wanted keys 2026-02-22T23:23:29.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 librados: calling monclient init 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient: init 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f508806ff20 0x7f50881e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): picked mon.noname-b con 0x7f508806ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5088155eb0 0x7f508806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): picked mon.noname-c con 0x7f5088155eb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f508815c970 0x7f508806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): picked mon.noname-a con 0x7f508815c970 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f508815d340 con 0x7f508815c970 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f508815d1c0 con 0x7f508806ff20 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f508815d4c0 con 0x7f5088155eb0 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.900+0000 7f508d18f640 10 monclient(hunting): authenticate will time out at 1084.627689s 2026-02-22T23:23:29.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7fc447fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4500c99a0 0x7fc4501f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 10 librados: watch_flush enter 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 10 librados: watch_flush exit 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 20 client.4512.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 10 client.4512.objecter successfully canceled tick 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3908078e50 msgr2=0x7f390807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3908078e50 0x7f390807b310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39280048b0 tx=0x7f3928008040 comp rx=0 tx=0).stop 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 10 monclient: shutdown 2026-02-22T23:23:29.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f393815ed40 msgr2=0x7f393814beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f393815ed40 0x7f393814beb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f393400a950 tx=0x7f39381601e0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 shutdown_connections 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3908078e50 0x7f390807b310 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f393815ed40 0x7f393814beb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f39381f4370 0x7f393815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 --2- 10.20.193.6:0/1311347986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f39200054c0 0x7f3938157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 >> 10.20.193.6:0/1311347986 conn(0x7f39380ce730 msgr2=0x7f39380cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 shutdown_connections 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 -- 10.20.193.6:0/1311347986 wait complete. 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7f393f534640 1 librados: shutdown 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.901+0000 7fc447fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4500c99a0 0x7fc4501f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40002/0 (socket says 10.20.193.6:40002) 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 1 -- 10.20.193.6:0/2966485327 learned_addr learned my addr 10.20.193.6:0/2966485327 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 10 monclient(hunting): get_auth_request con 0x7fc4500c99a0 auth_method 0 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7f23ce7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d00c99a0 0x7f23d006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7f23ce7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d00c99a0 0x7f23d006e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40000/0 (socket says 10.20.193.6:40000) 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7f23ce7fc640 1 -- 10.20.193.6:0/999826147 learned_addr learned my addr 10.20.193.6:0/999826147 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7f23ce7fc640 10 monclient(hunting): get_auth_request con 0x7f23d00c99a0 auth_method 0 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7f23ce7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7f23ce7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7f23ce7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.902+0000 7fc447fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 1 -- 10.20.193.6:0/3989744301 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684004cf0 con 0x7f769c156f60 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: dump: 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.911 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 1 -- 10.20.193.6:0/3989744301 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7684007110 con 0x7f769c156f60 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 1 -- 10.20.193.6:0/3989744301 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76840074e0 con 0x7f769c156f60 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 5 monclient: authenticate success, global_id 4518 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 monclient: _renew_subs 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76997fa640 4 set_mon_vals no callback set 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c154620 con 0x7f769c156f60 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f767b7fe640 10 monclient: dump: 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.912 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 client.4518.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 monclient: _renew_subs 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c154be0 con 0x7f769c156f60 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 1 librados: init done 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.903+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c1551f0 con 0x7f769c156f60 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f23ce7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f23ce7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f23ce7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 1 -- 10.20.193.6:0/449808492 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0044070 con 0x7f2de01527f0 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: dump: 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.914 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 1 -- 10.20.193.6:0/449808492 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd00406f0 con 0x7f2de01527f0 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 1 -- 10.20.193.6:0/449808492 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd003c040 con 0x7f2de01527f0 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 5 monclient: authenticate success, global_id 4530 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcf7fe640 4 set_mon_vals no callback set 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 monclient: _renew_subs 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de00c7830 con 0x7f2de01527f0 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2dcdffb640 10 monclient: dump: 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.915 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 client.4530.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 monclient: _renew_subs 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de00c7df0 con 0x7f2de01527f0 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 1 librados: init done 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.904+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de0152bd0 con 0x7f2de01527f0 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 10 monclient(hunting): handle_auth_done global_id 4539 payload 824 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 1 -- 10.20.193.6:0/2966485327 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc45006ff20 msgr2=0x7fc4501f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc45006ff20 0x7fc4501f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 1 -- 10.20.193.6:0/2966485327 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc45015a410 msgr2=0x7fc4501f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc45015a410 0x7fc4501f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.905+0000 7fc447fff640 1 -- 10.20.193.6:0/2966485327 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4500cca40 con 0x7fc4500c99a0 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 10 monclient(hunting): handle_auth_done global_id 4542 payload 824 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 1 -- 10.20.193.6:0/999826147 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f23d006ff20 msgr2=0x7f23d01f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f23d006ff20 0x7f23d01f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 1 -- 10.20.193.6:0/999826147 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f23d015a5e0 msgr2=0x7f23d01f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f23d015a5e0 0x7f23d01f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 1 monclient: found mon.noname-a 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:29.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.907+0000 7f23ce7fc640 1 -- 10.20.193.6:0/999826147 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d014c9f0 con 0x7f23d00c99a0 2026-02-22T23:23:29.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.7 2026-02-22T23:23:29.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.910+0000 7fc447fff640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4500c99a0 0x7fc4501f7170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b780 tx=0x7fc43800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.910+0000 7fc4457fa640 10 client.?.objecter ms_handle_connect 0x7fc4500c99a0 2026-02-22T23:23:29.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.910+0000 7fc4457fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.910+0000 7f23ce7fc640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d00c99a0 0x7f23d006e540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b570 tx=0x7f23b800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.910+0000 7f23af7fe640 10 client.?.objecter ms_handle_connect 0x7f23d00c99a0 2026-02-22T23:23:29.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.910+0000 7f23af7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.926+0000 7f31eeffd640 1 -- 10.20.193.6:0/1170235400 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f31f801c400 con 0x7f32080ca2d0 2026-02-22T23:23:29.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.926+0000 7f31eeffd640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f31d4078e00 0x7f31d407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.926+0000 7f3206575640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f31d4078e00 0x7f31d407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.926+0000 7f3206575640 10 monclient: get_auth_request con 0x7f31d4078e00 auth_method 0 2026-02-22T23:23:29.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.927+0000 7f3206575640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f31d4078e00 0x7f31d407b2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31fc039000 tx=0x7f31fc031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.927+0000 7f31eeffd640 10 client.4537.objecter ms_handle_connect 0x7f31d4078e00 2026-02-22T23:23:29.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.927+0000 7f31eeffd640 1 -- 10.20.193.6:0/1170235400 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f31f805e4e0 con 0x7f32080ca2d0 2026-02-22T23:23:29.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.927+0000 7f31eeffd640 10 client.4537.objecter ms_dispatch2 0x7f3208000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.927+0000 7f31eeffd640 3 client.4537.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.927+0000 7f31eeffd640 3 client.4537.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.927+0000 7f31eeffd640 20 client.4537.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.937+0000 7f31eeffd640 1 -- 10.20.193.6:0/1170235400 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f31f8021020 con 0x7f32080ca2d0 2026-02-22T23:23:29.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.937+0000 7f31eeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.937+0000 7f31eeffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:29.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.940+0000 7f95c18f5640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:29.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.940+0000 7f95c18f5640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:29.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.940+0000 7f95c18f5640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:29.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.940+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f95bc074cc0 con 0x7f95bc158c30 2026-02-22T23:23:29.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.950+0000 7f5086d76640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f508806ff20 0x7f50881e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7f5086d76640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f508806ff20 0x7f50881e9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36744/0 (socket says 10.20.193.6:36744) 2026-02-22T23:23:29.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7f5086d76640 1 -- 10.20.193.6:0/617064811 learned_addr learned my addr 10.20.193.6:0/617064811 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:29.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7f5086d76640 10 monclient(hunting): get_auth_request con 0x7f508806ff20 auth_method 0 2026-02-22T23:23:29.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7f5086d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:29.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7f5086d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:29.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7f5086d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:29.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7fafd67fc640 1 -- 10.20.193.6:0/1991861796 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fafd8065d90 con 0x7fafe80d1ea0 2026-02-22T23:23:29.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7fafd67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:29.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.951+0000 7fafd67fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:29.953 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:29.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.952+0000 7f5086d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:29.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.952+0000 7f5086d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:29.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.952+0000 7f5086d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:29.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 10 librados: watch_flush enter 2026-02-22T23:23:29.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 10 librados: watch_flush exit 2026-02-22T23:23:29.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 20 client.4531.objecter shutdown clearing up homeless session... 2026-02-22T23:23:29.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 10 client.4531.objecter successfully canceled tick 2026-02-22T23:23:29.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 10 monclient(hunting): handle_auth_done global_id 4543 payload 824 2026-02-22T23:23:29.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafb8078e30 msgr2=0x7fafb807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafb8078e30 0x7fafb807b2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fafe80cfc50 tx=0x7fafdc006cf0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 10 monclient: shutdown 2026-02-22T23:23:29.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:29.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 msgr2=0x7fafe80d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 1 -- 10.20.193.6:0/617064811 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5088155eb0 msgr2=0x7f508806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5088155eb0 0x7f508806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80d0da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafd800c4d0 tx=0x7fafd800c9a0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 1 -- 10.20.193.6:0/617064811 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f508815c970 msgr2=0x7f508806dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:29.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f508815c970 0x7f508806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 10 monclient: _finish_hunting 0 2026-02-22T23:23:29.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 1 monclient: found mon.noname-b 2026-02-22T23:23:29.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:29.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7f5086d76640 1 -- 10.20.193.6:0/617064811 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50881e7ed0 con 0x7f508806ff20 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 shutdown_connections 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafb8078e30 0x7fafb807b2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fafe815fa30 0x7fafe81ea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fafe815f650 0x7fafe81e9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 --2- 10.20.193.6:0/1991861796 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fafe80d1ea0 0x7fafe80d0da0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 >> 10.20.193.6:0/1991861796 conn(0x7fafe8072fc0 msgr2=0x7fafe8150f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 shutdown_connections 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 -- 10.20.193.6:0/1991861796 wait complete. 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.953+0000 7fafefd1b640 1 librados: shutdown 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.955+0000 7f5086d76640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f508806ff20 0x7f50881e9bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f507c00b4d0 tx=0x7f507c00b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.955+0000 7f50677fe640 10 client.?.objecter ms_handle_connect 0x7f508806ff20 2026-02-22T23:23:29.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.955+0000 7f50677fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 1 -- 10.20.193.6:0/617064811 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c004480 con 0x7f508806ff20 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: dump: 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.960 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 1 -- 10.20.193.6:0/617064811 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c015070 con 0x7f508806ff20 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 1 -- 10.20.193.6:0/617064811 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c002920 con 0x7f508806ff20 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 5 monclient: authenticate success, global_id 4543 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 10 monclient: _renew_subs 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f5085573640 4 set_mon_vals no callback set 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881e81b0 con 0x7f508806ff20 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:29.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f50677fe640 10 monclient: dump: 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 10 client.4543.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 10 monclient: _renew_subs 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508806d160 con 0x7f508806ff20 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.958+0000 7f508d18f640 1 librados: init done 2026-02-22T23:23:29.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.959+0000 7f508d18f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.959+0000 7f508d18f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.959+0000 7f508d18f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:29.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.959+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50881e86d0 con 0x7f508806ff20 2026-02-22T23:23:29.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.12 2026-02-22T23:23:29.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.969+0000 7f50677fe640 1 -- 10.20.193.6:0/617064811 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f507c010480 con 0x7f508806ff20 2026-02-22T23:23:29.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.970+0000 7f50677fe640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5054078e50 0x7f505407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.970+0000 7f5086575640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5054078e50 0x7f505407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.970+0000 7f5086575640 10 monclient: get_auth_request con 0x7f5054078e50 auth_method 0 2026-02-22T23:23:29.972 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.971+0000 7f50677fe640 10 client.4543.objecter ms_handle_connect 0x7f5054078e50 2026-02-22T23:23:29.972 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.971+0000 7f5086575640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5054078e50 0x7f505407b310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50880c0e90 tx=0x7f5070031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.972 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.971+0000 7f50677fe640 1 -- 10.20.193.6:0/617064811 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f507c05e1c0 con 0x7f508806ff20 2026-02-22T23:23:29.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.971+0000 7f50677fe640 10 client.4543.objecter ms_dispatch2 0x7f5088000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.971+0000 7f50677fe640 3 client.4543.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.971+0000 7f50677fe640 3 client.4543.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.971+0000 7f50677fe640 20 client.4543.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.985 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.984+0000 7f6b567fc640 1 -- 10.20.193.6:0/3056384900 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6b5c003a70 con 0x7f6b6815f650 2026-02-22T23:23:29.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.984+0000 7f6b567fc640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6b38078e50 0x7f6b3807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:29.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.985+0000 7f6b6dd60640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6b38078e50 0x7f6b3807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:29.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.985+0000 7f6b6dd60640 10 monclient: get_auth_request con 0x7f6b38078e50 auth_method 0 2026-02-22T23:23:29.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.985+0000 7f6b6dd60640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6b38078e50 0x7f6b3807b310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b58002a30 tx=0x7f6b580079f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:29.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.985+0000 7f6b567fc640 10 client.4601.objecter ms_handle_connect 0x7f6b38078e50 2026-02-22T23:23:29.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.987+0000 7f6b567fc640 1 -- 10.20.193.6:0/3056384900 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f6b5c05e120 con 0x7f6b6815f650 2026-02-22T23:23:29.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.987+0000 7f6b567fc640 10 client.4601.objecter ms_dispatch2 0x7f6b68000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:29.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.987+0000 7f6b567fc640 3 client.4601.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:29.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.987+0000 7f6b567fc640 3 client.4601.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:29.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.987+0000 7f6b567fc640 20 client.4601.objecter dump_active .. 0 homeless 2026-02-22T23:23:29.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.993+0000 7f6b567fc640 1 -- 10.20.193.6:0/3056384900 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6b680ca7a0 con 0x7f6b6815f650 2026-02-22T23:23:29.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.993+0000 7f6b567fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:29.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.993+0000 7f6b567fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:30.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.999+0000 7f9593fff640 1 -- 10.20.193.6:0/3586189641 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f95ac062470 con 0x7f95bc158c30 2026-02-22T23:23:30.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.999+0000 7f9593fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:30.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:29.999+0000 7f9593fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.001 INFO:teuthology.orchestra.run.trial006.stdout:94489280515 2026-02-22T23:23:30.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 10 librados: watch_flush enter 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 10 librados: watch_flush exit 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 20 client.4589.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 10 client.4589.objecter successfully canceled tick 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f958c078e50 msgr2=0x7f958c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f958c078e50 0x7f958c07b310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f95b0004800 tx=0x7f95b0009110 comp rx=0 tx=0).stop 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 10 monclient: shutdown 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95bc158c30 msgr2=0x7f95bc1590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95bc158c30 0x7f95bc1590e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0ce270 tx=0x7f95ac00a430 comp rx=0 tx=0).stop 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 shutdown_connections 2026-02-22T23:23:30.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f958c078e50 0x7f958c07b310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f95bc158c30 0x7f95bc1590e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f95bc0c9d10 0x7f95bc0ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 --2- 10.20.193.6:0/3586189641 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f95a8004900 0x7f95bc0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 >> 10.20.193.6:0/3586189641 conn(0x7f95bc0ce730 msgr2=0x7f95bc06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 shutdown_connections 2026-02-22T23:23:30.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 -- 10.20.193.6:0/3586189641 wait complete. 2026-02-22T23:23:30.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.001+0000 7f95c18f5640 1 librados: shutdown 2026-02-22T23:23:30.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.3 2026-02-22T23:23:30.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.023+0000 7f320cd33640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:30.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.023+0000 7f320cd33640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:30.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.023+0000 7f320cd33640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:30.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.023+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f32080cd860 con 0x7f32080ca2d0 2026-02-22T23:23:30.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.071+0000 7f50677fe640 1 -- 10.20.193.6:0/617064811 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f507c00d360 con 0x7f508806ff20 2026-02-22T23:23:30.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.071+0000 7f50677fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.071+0000 7f50677fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:30.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.073+0000 7f31eeffd640 1 -- 10.20.193.6:0/1170235400 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f31f8062ba0 con 0x7f32080ca2d0 2026-02-22T23:23:30.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.073+0000 7f31eeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:30.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.073+0000 7f31eeffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.075 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 10 librados: watch_flush enter 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 10 librados: watch_flush exit 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 20 client.4537.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 10 client.4537.objecter successfully canceled tick 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f31d4078e00 msgr2=0x7f31d407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f31d4078e00 0x7f31d407b2c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f31fc039000 tx=0x7f31fc031000 comp rx=0 tx=0).stop 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 10 monclient: shutdown 2026-02-22T23:23:30.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32080ca2d0 msgr2=0x7f32081f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32080ca2d0 0x7f32081f5830 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f32080c88f0 tx=0x7f31f8004e90 comp rx=0 tx=0).stop 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 shutdown_connections 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f31d4078e00 0x7f31d407b2c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32080ca2d0 0x7f32081f5830 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.075+0000 7f320cd33640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32080c9e50 0x7f32081f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.076+0000 7f320cd33640 1 --2- 10.20.193.6:0/1170235400 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f31f4003ba0 0x7f3208156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.076+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 >> 10.20.193.6:0/1170235400 conn(0x7f32080ce730 msgr2=0x7f32080cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.076+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 shutdown_connections 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.076+0000 7f320cd33640 1 -- 10.20.193.6:0/1170235400 wait complete. 2026-02-22T23:23:30.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.076+0000 7f320cd33640 1 librados: shutdown 2026-02-22T23:23:30.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.079+0000 7f6b6ffeb640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:30.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.079+0000 7f6b6ffeb640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:30.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.079+0000 7f6b6ffeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:30.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.079+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6b68074ea0 con 0x7f6b6815f650 2026-02-22T23:23:30.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.081+0000 7f6b567fc640 1 -- 10.20.193.6:0/3056384900 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f6b5c0627e0 con 0x7f6b6815f650 2026-02-22T23:23:30.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.081+0000 7f6b567fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:30.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.081+0000 7f6b567fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.082 INFO:teuthology.orchestra.run.trial006.stdout:90194313218 2026-02-22T23:23:30.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.082+0000 7f6b6ffeb640 10 librados: watch_flush enter 2026-02-22T23:23:30.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.082+0000 7f6b6ffeb640 10 librados: watch_flush exit 2026-02-22T23:23:30.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.082+0000 7f6b6ffeb640 20 client.4601.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.082+0000 7f6b6ffeb640 10 client.4601.objecter successfully canceled tick 2026-02-22T23:23:30.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6b38078e50 msgr2=0x7f6b3807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6b38078e50 0x7f6b3807b310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6b58002a30 tx=0x7f6b580079f0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 10 monclient: shutdown 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b6815f650 msgr2=0x7f6b681f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b6815f650 0x7f6b681f3e20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c730 tx=0x7f6b5c00cc00 comp rx=0 tx=0).stop 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 shutdown_connections 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6b38078e50 0x7f6b3807b310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6b6815fa30 0x7f6b6814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6b6815f650 0x7f6b681f3e20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 --2- 10.20.193.6:0/3056384900 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6b680c9b00 0x7f6b68149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 >> 10.20.193.6:0/3056384900 conn(0x7f6b68072fc0 msgr2=0x7f6b6815ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 shutdown_connections 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 -- 10.20.193.6:0/3056384900 wait complete. 2026-02-22T23:23:30.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.083+0000 7f6b6ffeb640 1 librados: shutdown 2026-02-22T23:23:30.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.11 2026-02-22T23:23:30.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313218 for osd.10 2026-02-22T23:23:30.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.115+0000 7f767b7fe640 1 -- 10.20.193.6:0/3989744301 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f768400a5c0 con 0x7f769c156f60 2026-02-22T23:23:30.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.115+0000 7f767b7fe640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7668078e50 0x7f766807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.115+0000 7f769affd640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7668078e50 0x7f766807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.115+0000 7f769affd640 10 monclient: get_auth_request con 0x7f7668078e50 auth_method 0 2026-02-22T23:23:30.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.116+0000 7f767b7fe640 10 client.4518.objecter ms_handle_connect 0x7f7668078e50 2026-02-22T23:23:30.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.116+0000 7f769affd640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7668078e50 0x7f766807b310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76900097d0 tx=0x7f7690006db0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 1 -- 10.20.193.6:0/3989744301 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f7684010da0 con 0x7f769c156f60 2026-02-22T23:23:30.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 10 client.4518.objecter ms_dispatch2 0x7f769c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:30.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 3 client.4518.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:30.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 3 client.4518.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:30.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 20 client.4518.objecter dump_active .. 0 homeless 2026-02-22T23:23:30.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 1 -- 10.20.193.6:0/3989744301 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f76840627b0 con 0x7f769c156f60 2026-02-22T23:23:30.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.129+0000 7f767b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:30.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.138+0000 7f2dcdffb640 1 -- 10.20.193.6:0/449808492 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2dd003f050 con 0x7f2de01527f0 2026-02-22T23:23:30.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.139+0000 7f2dcdffb640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2db0078e50 0x7f2db007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.139+0000 7f2de493e640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2db0078e50 0x7f2db007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.139+0000 7f2de493e640 10 monclient: get_auth_request con 0x7f2db0078e50 auth_method 0 2026-02-22T23:23:30.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.139+0000 7f2de493e640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2db0078e50 0x7f2db007b310 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2de014ff90 tx=0x7f2dd4007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.139+0000 7f2dcdffb640 10 client.4530.objecter ms_handle_connect 0x7f2db0078e50 2026-02-22T23:23:30.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 1 -- 10.20.193.6:0/449808492 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f2dd002fac0 con 0x7f2de01527f0 2026-02-22T23:23:30.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 10 client.4530.objecter ms_dispatch2 0x7f2de0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:30.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 3 client.4530.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:30.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 3 client.4530.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:30.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 20 client.4530.objecter dump_active .. 0 homeless 2026-02-22T23:23:30.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 1 -- 10.20.193.6:0/449808492 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2dd0087590 con 0x7f2de01527f0 2026-02-22T23:23:30.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.147+0000 7f2dcdffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:30.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 1 -- 10.20.193.6:0/999826147 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8013020 con 0x7f23d00c99a0 2026-02-22T23:23:30.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: dump: 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.154 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 1 -- 10.20.193.6:0/999826147 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b80044a0 con 0x7f23d00c99a0 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 1 -- 10.20.193.6:0/999826147 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80029c0 con 0x7f23d00c99a0 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23cd7fa640 4 set_mon_vals no callback set 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23af7fe640 10 monclient: dump: 2026-02-22T23:23:30.155 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 5 monclient: authenticate success, global_id 4542 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 10 monclient: _renew_subs 2026-02-22T23:23:30.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d0154ff0 con 0x7f23d00c99a0 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 10 client.4542.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 10 monclient: _renew_subs 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d01555b0 con 0x7f23d00c99a0 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.148+0000 7f23d53e4640 1 librados: init done 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.149+0000 7f23d53e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.149+0000 7f23d53e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.149+0000 7f23d53e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.149+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d0155ea0 con 0x7f23d00c99a0 2026-02-22T23:23:30.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 1 -- 10.20.193.6:0/2966485327 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438004480 con 0x7fc4500c99a0 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: dump: 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.160 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 1 -- 10.20.193.6:0/2966485327 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc438010070 con 0x7fc4500c99a0 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 1 -- 10.20.193.6:0/2966485327 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438002920 con 0x7fc4500c99a0 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 5 monclient: authenticate success, global_id 4539 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc446ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:30.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 monclient: _renew_subs 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4501f5070 con 0x7fc4500c99a0 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc4457fa640 10 monclient: dump: 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.162 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 client.4539.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 monclient: _renew_subs 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4501f5630 con 0x7fc4500c99a0 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 1 librados: init done 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc450070300 con 0x7fc4500c99a0 2026-02-22T23:23:30.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.157+0000 7f508d18f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:30.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.158+0000 7f508d18f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:30.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.158+0000 7f508d18f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:30.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.158+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f508806cf50 con 0x7f508806ff20 2026-02-22T23:23:30.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.160+0000 7f50677fe640 1 -- 10.20.193.6:0/617064811 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f507c062880 con 0x7f508806ff20 2026-02-22T23:23:30.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.160+0000 7f50677fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:30.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.160+0000 7f50677fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.165 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:30.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 10 librados: watch_flush enter 2026-02-22T23:23:30.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 10 librados: watch_flush exit 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 20 client.4543.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 10 client.4543.objecter successfully canceled tick 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5054078e50 msgr2=0x7f505407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5054078e50 0x7f505407b310 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50880c0e90 tx=0x7f5070031000 comp rx=0 tx=0).stop 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 10 monclient: shutdown 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f508806ff20 msgr2=0x7f50881e9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f508806ff20 0x7f50881e9bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f507c00b4d0 tx=0x7f507c00b9a0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 shutdown_connections 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5054078e50 0x7f505407b310 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f508815c970 0x7f508806dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5088155eb0 0x7f508806d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 --2- 10.20.193.6:0/617064811 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f508806ff20 0x7f50881e9bd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 >> 10.20.193.6:0/617064811 conn(0x7f50880cd780 msgr2=0x7f50880cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 shutdown_connections 2026-02-22T23:23:30.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 -- 10.20.193.6:0/617064811 wait complete. 2026-02-22T23:23:30.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.162+0000 7f508d18f640 1 librados: shutdown 2026-02-22T23:23:30.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.165+0000 7f23af7fe640 1 -- 10.20.193.6:0/999826147 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f23b8002b60 con 0x7f23d00c99a0 2026-02-22T23:23:30.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.166+0000 7f23af7fe640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f239c078e80 0x7f239c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.166+0000 7f23ceffd640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f239c078e80 0x7f239c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.166+0000 7f23ceffd640 10 monclient: get_auth_request con 0x7f239c078e80 auth_method 0 2026-02-22T23:23:30.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.167+0000 7f23af7fe640 10 client.4542.objecter ms_handle_connect 0x7f239c078e80 2026-02-22T23:23:30.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.167+0000 7f23ceffd640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f239c078e80 0x7f239c07b340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009970 tx=0x7f23c4031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.169+0000 7f23af7fe640 1 -- 10.20.193.6:0/999826147 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f23b805e040 con 0x7f23d00c99a0 2026-02-22T23:23:30.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.169+0000 7f23af7fe640 10 client.4542.objecter ms_dispatch2 0x7f23d0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:30.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.169+0000 7f23af7fe640 3 client.4542.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:30.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.169+0000 7f23af7fe640 3 client.4542.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:30.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.169+0000 7f23af7fe640 20 client.4542.objecter dump_active .. 0 homeless 2026-02-22T23:23:30.173 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.6 2026-02-22T23:23:30.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.215+0000 7f76a13e4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:30.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.215+0000 7f76a13e4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:30.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.215+0000 7f76a13e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.215+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f769c1573f0 con 0x7f769c156f60 2026-02-22T23:23:30.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.235+0000 7f2de73ca640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:30.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.235+0000 7f2de73ca640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:30.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.235+0000 7f2de73ca640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.235+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7f2de00d0240 con 0x7f2de01527f0 2026-02-22T23:23:30.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.319+0000 7f23af7fe640 1 -- 10.20.193.6:0/999826147 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f23b801d020 con 0x7f23d00c99a0 2026-02-22T23:23:30.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.319+0000 7f23af7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.319+0000 7f23af7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:30.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.339+0000 7fc4457fa640 1 -- 10.20.193.6:0/2966485327 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc43801d030 con 0x7fc4500c99a0 2026-02-22T23:23:30.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.340+0000 7fc4457fa640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc420078e50 0x7fc42007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.340+0000 7fc454a04640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc420078e50 0x7fc42007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.340+0000 7fc454a04640 10 monclient: get_auth_request con 0x7fc420078e50 auth_method 0 2026-02-22T23:23:30.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.340+0000 7fc4457fa640 10 client.4539.objecter ms_handle_connect 0x7fc420078e50 2026-02-22T23:23:30.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.340+0000 7fc454a04640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc420078e50 0x7fc42007b310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c5380 tx=0x7fc440031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.341+0000 7fc4457fa640 1 -- 10.20.193.6:0/2966485327 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fc43801dde0 con 0x7fc4500c99a0 2026-02-22T23:23:30.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.341+0000 7fc4457fa640 10 client.4539.objecter ms_dispatch2 0x7fc450000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:30.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.341+0000 7fc4457fa640 3 client.4539.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:30.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.341+0000 7fc4457fa640 3 client.4539.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:30.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.341+0000 7fc4457fa640 20 client.4539.objecter dump_active .. 0 homeless 2026-02-22T23:23:30.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.344+0000 7f767b7fe640 1 -- 10.20.193.6:0/3989744301 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f76840620b0 con 0x7f769c156f60 2026-02-22T23:23:30.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.344+0000 7f767b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:30.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.344+0000 7f767b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.346 INFO:teuthology.orchestra.run.trial006.stdout:98784247811 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 10 librados: watch_flush enter 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 10 librados: watch_flush exit 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 20 client.4518.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 10 client.4518.objecter successfully canceled tick 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7668078e50 msgr2=0x7f766807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7668078e50 0x7f766807b310 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f76900097d0 tx=0x7f7690006db0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 10 monclient: shutdown 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f769c156f60 msgr2=0x7f769c06e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f769c156f60 0x7f769c06e210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f768400ca70 tx=0x7f768400cf40 comp rx=0 tx=0).stop 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 shutdown_connections 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7668078e50 0x7f766807b310 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f769c06e750 0x7f769c1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f769c156f60 0x7f769c06e210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 --2- 10.20.193.6:0/3989744301 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7688005510 0x7f769c1f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 >> 10.20.193.6:0/3989744301 conn(0x7f769c0cd780 msgr2=0x7f769c073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 shutdown_connections 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.346+0000 7f76a13e4640 1 -- 10.20.193.6:0/3989744301 wait complete. 2026-02-22T23:23:30.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.347+0000 7f76a13e4640 1 librados: shutdown 2026-02-22T23:23:30.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.0 2026-02-22T23:23:30.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.410+0000 7f23d53e4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:30.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.410+0000 7f23d53e4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:30.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.410+0000 7f23d53e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.410+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f23d0070300 con 0x7f23d00c99a0 2026-02-22T23:23:30.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.453+0000 7fc4457fa640 1 -- 10.20.193.6:0/2966485327 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc438024080 con 0x7fc4500c99a0 2026-02-22T23:23:30.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.453+0000 7fc4457fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.453+0000 7fc4457fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:30.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.455+0000 7f23af7fe640 1 -- 10.20.193.6:0/999826147 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f23d014c9f0 con 0x7f23d00c99a0 2026-02-22T23:23:30.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.455+0000 7f23af7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:30.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.455+0000 7f23af7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.457 INFO:teuthology.orchestra.run.trial006.stdout:94489280515 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 10 librados: watch_flush enter 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 10 librados: watch_flush exit 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 20 client.4542.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 10 client.4542.objecter successfully canceled tick 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f239c078e80 msgr2=0x7f239c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f239c078e80 0x7f239c07b340 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009970 tx=0x7f23c4031000 comp rx=0 tx=0).stop 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 10 monclient: shutdown 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d00c99a0 msgr2=0x7f23d006e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.456+0000 7f23d53e4640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d00c99a0 0x7f23d006e540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b570 tx=0x7f23b800ba40 comp rx=0 tx=0).stop 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 shutdown_connections 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f239c078e80 0x7f239c07b340 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f23d015a5e0 0x7f23d01f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f23d00c99a0 0x7f23d006e540 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 --2- 10.20.193.6:0/999826147 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f23d006ff20 0x7f23d01f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 >> 10.20.193.6:0/999826147 conn(0x7f23d00cd780 msgr2=0x7f23d006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 shutdown_connections 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 -- 10.20.193.6:0/999826147 wait complete. 2026-02-22T23:23:30.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.457+0000 7f23d53e4640 1 librados: shutdown 2026-02-22T23:23:30.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.5 2026-02-22T23:23:30.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.484+0000 7f2dcdffb640 1 -- 10.20.193.6:0/449808492 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f2dd0086e90 con 0x7f2de01527f0 2026-02-22T23:23:30.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.484+0000 7f2dcdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:30.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.484+0000 7f2dcdffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.486 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:30.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 10 librados: watch_flush enter 2026-02-22T23:23:30.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 10 librados: watch_flush exit 2026-02-22T23:23:30.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 20 client.4530.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 10 client.4530.objecter successfully canceled tick 2026-02-22T23:23:30.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2db0078e50 msgr2=0x7f2db007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2db0078e50 0x7f2db007b310 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2de014ff90 tx=0x7f2dd4007ab0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 10 monclient: shutdown 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2de01527f0 msgr2=0x7f2de01f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2de01527f0 0x7f2de01f75c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009a70 tx=0x7f2dd00047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 shutdown_connections 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2db0078e50 0x7f2db007b310 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2de015d730 0x7f2de00c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2de015d2c0 0x7f2de00ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 --2- 10.20.193.6:0/449808492 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2de01527f0 0x7f2de01f75c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 >> 10.20.193.6:0/449808492 conn(0x7f2de00ce730 msgr2=0x7f2de00c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 shutdown_connections 2026-02-22T23:23:30.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 -- 10.20.193.6:0/449808492 wait complete. 2026-02-22T23:23:30.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.486+0000 7f2de73ca640 1 librados: shutdown 2026-02-22T23:23:30.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280516 for osd.15 2026-02-22T23:23:30.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.539+0000 7fc456c8f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-22T23:23:30.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.539+0000 7fc456c8f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-22T23:23:30.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.539+0000 7fc456c8f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.539+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc4501f61e0 con 0x7fc4500c99a0 2026-02-22T23:23:30.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.541+0000 7fc4457fa640 1 -- 10.20.193.6:0/2966485327 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc438061a00 con 0x7fc4500c99a0 2026-02-22T23:23:30.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.541+0000 7fc4457fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2026-02-22T23:23:30.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.541+0000 7fc4457fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:30.543 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 10 librados: watch_flush enter 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 10 librados: watch_flush exit 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 20 client.4539.objecter shutdown clearing up homeless session... 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 10 client.4539.objecter successfully canceled tick 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc420078e50 msgr2=0x7fc42007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc420078e50 0x7fc42007b310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc4500c5380 tx=0x7fc440031000 comp rx=0 tx=0).stop 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 10 monclient: shutdown 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4500c99a0 msgr2=0x7fc4501f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4500c99a0 0x7fc4501f7170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b780 tx=0x7fc43800bc50 comp rx=0 tx=0).stop 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 shutdown_connections 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc420078e50 0x7fc42007b310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc45015a410 0x7fc4501f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4500c99a0 0x7fc4501f7170 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 --2- 10.20.193.6:0/2966485327 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc45006ff20 0x7fc4501f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 >> 10.20.193.6:0/2966485327 conn(0x7fc4500cd780 msgr2=0x7fc450072480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 shutdown_connections 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.542+0000 7fc456c8f640 1 -- 10.20.193.6:0/2966485327 wait complete. 2026-02-22T23:23:30.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.543+0000 7fc456c8f640 1 librados: shutdown 2026-02-22T23:23:30.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.2 2026-02-22T23:23:30.552 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:30.701 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.769+0000 7fd0fad1b640 10 monclient: get_monmap_and_config 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.769+0000 7fd0fad1b640 10 monclient: build_initial_monmap 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.769+0000 7fd0fad1b640 10 monclient: monmap: 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:30.769846+0000 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:30.769846+0000 2026-02-22T23:23:30.771 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.769+0000 7fd0fad1b640 1 Processor -- start 2026-02-22T23:23:30.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.769+0000 7fd0fad1b640 1 -- start start 2026-02-22T23:23:30.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.769+0000 7fd0fad1b640 10 monclient: init 2026-02-22T23:23:30.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:30.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient: _add_conns ranks=[3,5,4,2,0,1] 2026-02-22T23:23:30.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd0f41527d0 0x7fd0f406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): picked mon.noname-a con 0x7fd0f41527d0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:30.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd0f4152bb0 0x7fd0f40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): picked mon.noname-c con 0x7fd0f4152bb0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:30.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fd0f40c5fc0 0x7fd0f40c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): picked mon.noname-b con 0x7fd0f40c5fc0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f406f1f0 con 0x7fd0f41527d0 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f40c6470 con 0x7fd0f40c5fc0 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f406e300 con 0x7fd0f4152bb0 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0fad1b640 10 monclient(hunting): authenticate will time out at 1085.497238s 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0f8a90640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd0f41527d0 0x7fd0f406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.770+0000 7fd0ebfff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd0f4152bb0 0x7fd0f40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd0f4152bb0 0x7fd0f40c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58828/0 (socket says 10.20.193.6:58828) 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd0f41527d0 0x7fd0f406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40026/0 (socket says 10.20.193.6:40026) 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 1 -- 10.20.193.6:0/1883258327 learned_addr learned my addr 10.20.193.6:0/1883258327 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 10 monclient(hunting): get_auth_request con 0x7fd0f4152bb0 auth_method 0 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 10 monclient(hunting): get_auth_request con 0x7fd0f41527d0 auth_method 0 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 1 --2- 10.20.193.6:0/1883258327 >> v2:10.20.193.40:3300/0 conn(0x7fd0f40c5fc0 0x7fd0f40c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 10 monclient(hunting): get_auth_request con 0x7fd0f40c5fc0 auth_method 0 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f8a90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.771+0000 7fd0f9291640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 10 monclient(hunting): handle_auth_done global_id 4545 payload 274 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 1 -- 10.20.193.6:0/1883258327 >> v2:10.20.193.80:3300/0 conn(0x7fd0f4152bb0 msgr2=0x7fd0f40c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 1 --2- 10.20.193.6:0/1883258327 >> v2:10.20.193.80:3300/0 conn(0x7fd0f4152bb0 0x7fd0f40c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 1 -- 10.20.193.6:0/1883258327 >> v2:10.20.193.40:3300/0 conn(0x7fd0f40c5fc0 msgr2=0x7fd0f40c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 1 --2- 10.20.193.6:0/1883258327 >> v2:10.20.193.40:3300/0 conn(0x7fd0f40c5fc0 0x7fd0f40c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 10 monclient: _finish_hunting 0 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 1 monclient: found mon.noname-a 2026-02-22T23:23:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.772+0000 7fd0f8a90640 1 -- 10.20.193.6:0/1883258327 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f406e5e0 con 0x7fd0f41527d0 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.773+0000 7fd0f8a90640 1 --2- 10.20.193.6:0/1883258327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f406da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4158940 tx=0x7fd0e402edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d5ffbd1bbd107d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 1 -- 10.20.193.6:0/1883258327 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e403c070 con 0x7fd0f41527d0 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: dump: 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.788 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 1 -- 10.20.193.6:0/1883258327 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e402fa80 con 0x7fd0f41527d0 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 5 monclient: authenticate success, global_id 4545 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 1 -- 10.20.193.6:0/1883258327 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e402fe50 con 0x7fd0f41527d0 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0eb7fe640 10 monclient: dump: 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 4 set_mon_vals no callback set 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 10 monclient: shutdown 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 -- 10.20.193.6:0/1883258327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 msgr2=0x7fd0f406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/1883258327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f406da60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4158940 tx=0x7fd0e402edc0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 -- 10.20.193.6:0/1883258327 shutdown_connections 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/1883258327 >> v2:10.20.193.40:3300/0 conn(0x7fd0f40c5fc0 0x7fd0f40c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/1883258327 >> v2:10.20.193.80:3300/0 conn(0x7fd0f4152bb0 0x7fd0f40c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/1883258327 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f406da60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 -- 10.20.193.6:0/1883258327 >> 10.20.193.6:0/1883258327 conn(0x7fd0f40cd780 msgr2=0x7fd0f40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 -- 10.20.193.6:0/1883258327 shutdown_connections 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 1 -- 10.20.193.6:0/1883258327 wait complete. 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 10 monclient: build_initial_monmap 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.775+0000 7fd0fad1b640 10 monclient: monmap: 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 librados: starting msgr at 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 librados: starting objecter 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 Processor -- start 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 -- start start 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 librados: setting wanted keys 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 librados: calling monclient init 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient: init 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd0f40c5fc0 0x7fd0f41f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): picked mon.noname-b con 0x7fd0f40c5fc0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f4157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): picked mon.noname-a con 0x7fd0f41527d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd0f4152bb0 0x7fd0f41f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): picked mon.noname-c con 0x7fd0f4152bb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f406ee80 con 0x7fd0f41527d0 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f406eba0 con 0x7fd0f40c5fc0 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f406dfa0 con 0x7fd0f4152bb0 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0fad1b640 10 monclient(hunting): authenticate will time out at 1085.503657s 2026-02-22T23:23:30.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.776+0000 7fd0ebfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f4157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40040/0 (socket says 10.20.193.6:40040) 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd0f4152bb0 0x7fd0f41f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 1 -- 10.20.193.6:0/844521614 learned_addr learned my addr 10.20.193.6:0/844521614 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 10 monclient(hunting): get_auth_request con 0x7fd0f41527d0 auth_method 0 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd0f40c5fc0 0x7fd0f41f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0ebfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 10 monclient(hunting): get_auth_request con 0x7fd0f4152bb0 auth_method 0 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 10 monclient(hunting): get_auth_request con 0x7fd0f40c5fc0 auth_method 0 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f9291640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.777+0000 7fd0f8a90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 10 monclient(hunting): handle_auth_done global_id 4548 payload 824 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 1 -- 10.20.193.6:0/844521614 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd0f4152bb0 msgr2=0x7fd0f41f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd0f4152bb0 0x7fd0f41f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 1 -- 10.20.193.6:0/844521614 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd0f40c5fc0 msgr2=0x7fd0f41f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd0f40c5fc0 0x7fd0f41f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 1 monclient: found mon.noname-a 2026-02-22T23:23:30.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.778+0000 7fd0ebfff640 1 -- 10.20.193.6:0/844521614 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f41f4a10 con 0x7fd0f41527d0 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.779+0000 7fd0ebfff640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f4157150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00e8c0 tx=0x7fd0dc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.779+0000 7fd0e97fa640 10 client.?.objecter ms_handle_connect 0x7fd0f41527d0 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.779+0000 7fd0e97fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0e97fa640 1 -- 10.20.193.6:0/844521614 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc017070 con 0x7fd0f41527d0 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0e97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0e97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0e97fa640 10 monclient: dump: 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.798 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0e97fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0e97fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0e97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 5 monclient: authenticate success, global_id 4548 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 10 monclient: _renew_subs 2026-02-22T23:23:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f4157b90 con 0x7fd0f41527d0 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 10 client.4548.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 10 monclient: _renew_subs 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f4158150 con 0x7fd0f41527d0 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.780+0000 7fd0fad1b640 1 librados: init done 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0e97fa640 1 -- 10.20.193.6:0/844521614 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc005a90 con 0x7fd0f41527d0 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0e97fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0e97fa640 1 -- 10.20.193.6:0/844521614 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc021060 con 0x7fd0f41527d0 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0e97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0e97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0e97fa640 10 monclient: dump: 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0eaffd640 4 set_mon_vals no callback set 2026-02-22T23:23:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0fad1b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0fad1b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0fad1b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.781+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f406da60 con 0x7fd0f41527d0 2026-02-22T23:23:30.824 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2026-02-22T23:23:30.867 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-22T23:23:30.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.884+0000 7fd0e97fa640 1 -- 10.20.193.6:0/844521614 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd0dc024060 con 0x7fd0f41527d0 2026-02-22T23:23:30.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.884+0000 7fd0e97fa640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd0c4078e30 0x7fd0c407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.884+0000 7fd0f8a90640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd0c4078e30 0x7fd0c407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.884+0000 7fd0f8a90640 10 monclient: get_auth_request con 0x7fd0c4078e30 auth_method 0 2026-02-22T23:23:30.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.885+0000 7fd0f8a90640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd0c4078e30 0x7fd0c407b2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4039000 tx=0x7fd0e4033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.885+0000 7fd0e97fa640 10 client.4548.objecter ms_handle_connect 0x7fd0c4078e30 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7f5b90e2b640 10 monclient: get_monmap_and_config 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7f5b90e2b640 10 monclient: build_initial_monmap 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7f5b90e2b640 10 monclient: monmap: 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:30.893843+0000 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:30.893843+0000 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.894 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 1 -- 10.20.193.6:0/844521614 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fd0dc097490 con 0x7fd0f41527d0 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 10 client.4548.objecter ms_dispatch2 0x7fd0f4000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 3 client.4548.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 3 client.4548.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 20 client.4548.objecter dump_active .. 0 homeless 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 1 -- 10.20.193.6:0/844521614 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd0dc061c60 con 0x7fd0f41527d0 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7fd0e97fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:30.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7f5b90e2b640 1 Processor -- start 2026-02-22T23:23:30.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7f5b90e2b640 1 -- start start 2026-02-22T23:23:30.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.893+0000 7f5b90e2b640 10 monclient: init 2026-02-22T23:23:30.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:30.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient: _add_conns ranks=[2,5,1,3,0,4] 2026-02-22T23:23:30.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5b8c06ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f5b8c0c99a0 0x7f5b8c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient(hunting): picked mon.noname-c con 0x7f5b8c0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5b8c15a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c0c7e90 con 0x7f5b8c15a5e0 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c0c8170 con 0x7f5b8c06ff20 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c159a40 con 0x7f5b8c0c99a0 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.894+0000 7f5b90e2b640 10 monclient(hunting): authenticate will time out at 1085.621398s 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b8ad76640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f5b8c15a5e0 0x7f5b8c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60396/0 (socket says 10.20.193.6:60396) 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b8ad76640 1 -- 10.20.193.6:0/1201365241 learned_addr learned my addr 10.20.193.6:0/1201365241 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 1 --2- 10.20.193.6:0/1201365241 >> v2:10.20.193.80:3300/0 conn(0x7f5b8c0c99a0 0x7f5b8c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 10 monclient(hunting): get_auth_request con 0x7f5b8c0c99a0 auth_method 0 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.895+0000 7f5b89d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.896+0000 7f5b89573640 1 -- 10.20.193.6:0/1201365241 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2601623655 0 0) 0x7f5b8c0c7e90 con 0x7f5b8c15a5e0 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.896+0000 7f5b89573640 10 monclient(hunting): _negotiate 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.896+0000 7f5b89573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.896+0000 7f5b89573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.896+0000 7f5b89573640 10 monclient(hunting): my global_id is 4555 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.896+0000 7f5b89573640 1 -- 10.20.193.6:0/1201365241 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b78003620 con 0x7f5b8c15a5e0 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 10 monclient(hunting): handle_auth_done global_id 4613 payload 274 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 1 -- 10.20.193.6:0/1201365241 >> v1:10.20.193.80:6789/0 conn(0x7f5b8c06ff20 legacy=0x7f5b8c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 1 -- 10.20.193.6:0/1201365241 >> v1:10.20.193.40:6789/0 conn(0x7f5b8c15a5e0 legacy=0x7f5b8c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 10 monclient: _finish_hunting 0 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 1 monclient: found mon.noname-c 2026-02-22T23:23:30.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:30.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:30.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.897+0000 7f5b89d74640 1 -- 10.20.193.6:0/1201365241 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c0c8450 con 0x7f5b8c0c99a0 2026-02-22T23:23:30.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.907+0000 7f5b89d74640 1 --2- 10.20.193.6:0/1201365241 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c0c99a0 0x7f5b8c070300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b740098d0 tx=0x7f5b7402eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f98cfb1cc7443f7f server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.919 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-22T23:23:30.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.935+0000 7f05cd224640 10 monclient: get_monmap_and_config 2026-02-22T23:23:30.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.935+0000 7f05cd224640 10 monclient: build_initial_monmap 2026-02-22T23:23:30.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.935+0000 7f05cd224640 10 monclient: monmap: 2026-02-22T23:23:30.936 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:30.936054+0000 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:30.936054+0000 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:30.937 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.935+0000 7f05cd224640 1 Processor -- start 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.935+0000 7f05cd224640 1 -- start start 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.935+0000 7f05cd224640 10 monclient: init 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient: _add_conns ranks=[3,0,2,5,1,4] 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f05c806ff20 0x7f05c8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient(hunting): picked mon.noname-a con 0x7f05c806ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f05c815c970 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f05c8155eb0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c80725b0 con 0x7f05c815c970 2026-02-22T23:23:30.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c8072890 con 0x7f05c8155eb0 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f05c8072b70 con 0x7f05c806ff20 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.936+0000 7f05cd224640 10 monclient(hunting): authenticate will time out at 1085.663658s 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.937+0000 7f05c6d76640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f05c806ff20 0x7f05c8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.937+0000 7f05c7577640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f05c8155eb0 0x7f05c80c7d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36274/0 (socket says 10.20.193.6:36274) 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.937+0000 7f05c7577640 1 -- 10.20.193.6:0/2856897350 learned_addr learned my addr 10.20.193.6:0/2856897350 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.937+0000 7f05c6d76640 10 monclient(hunting): get_auth_request con 0x7f05c806ff20 auth_method 0 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.937+0000 7f05c6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.937+0000 7f05c6d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.937+0000 7f05c6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.938+0000 7f05c6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.938+0000 7f05c6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.938+0000 7f05c6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 10 monclient(hunting): handle_auth_done global_id 4551 payload 274 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 1 -- 10.20.193.6:0/2856897350 >> v1:10.20.193.80:6789/0 conn(0x7f05c8155eb0 legacy=0x7f05c80c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 1 -- 10.20.193.6:0/2856897350 >> v1:10.20.193.6:6789/0 conn(0x7f05c815c970 legacy=0x7f05c81557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 10 monclient: _finish_hunting 0 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 1 monclient: found mon.noname-a 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:30.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.939+0000 7f05c6d76640 1 -- 10.20.193.6:0/2856897350 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c8072e50 con 0x7f05c806ff20 2026-02-22T23:23:30.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.941+0000 7f05c6d76640 1 --2- 10.20.193.6:0/2856897350 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c806ff20 0x7f05c8070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0098a0 tx=0x7f05bc02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7065d2a29839c602 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 1 -- 10.20.193.6:0/2856897350 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc03c070 con 0x7f05c806ff20 2026-02-22T23:23:30.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:30.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: dump: 2026-02-22T23:23:30.945 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.945 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: _finish_auth 0 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: _check_auth_tickets 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 1 -- 10.20.193.6:0/2856897350 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05bc02fb70 con 0x7f05c806ff20 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 1 -- 10.20.193.6:0/2856897350 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc037440 con 0x7f05c806ff20 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05cd224640 5 monclient: authenticate success, global_id 4551 2026-02-22T23:23:30.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05cd224640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05cd224640 4 set_mon_vals no callback set 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.943+0000 7f05c5d74640 10 monclient: dump: 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 10 monclient: shutdown 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 1 -- 10.20.193.6:0/2856897350 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c806ff20 msgr2=0x7f05c8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 1 --2- 10.20.193.6:0/2856897350 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c806ff20 0x7f05c8070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0098a0 tx=0x7f05bc02ee60 comp rx=0 tx=0).stop 2026-02-22T23:23:30.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 1 -- 10.20.193.6:0/2856897350 shutdown_connections 2026-02-22T23:23:30.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 1 --2- 10.20.193.6:0/2856897350 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c806ff20 0x7f05c8070300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 1 -- 10.20.193.6:0/2856897350 >> 10.20.193.6:0/2856897350 conn(0x7f05c80cd780 msgr2=0x7f05c80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 1 -- 10.20.193.6:0/2856897350 shutdown_connections 2026-02-22T23:23:30.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 1 -- 10.20.193.6:0/2856897350 wait complete. 2026-02-22T23:23:30.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 10 monclient: build_initial_monmap 2026-02-22T23:23:30.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.944+0000 7f05cd224640 10 monclient: monmap: 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 librados: starting msgr at 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 librados: starting objecter 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 Processor -- start 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 -- start start 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 librados: setting wanted keys 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 librados: calling monclient init 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient: init 2026-02-22T23:23:30.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f05c806ff20 0x7f05c8147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): picked mon.noname-c con 0x7f05c806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c8155eb0 0x7f05c8152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): picked mon.noname-a con 0x7f05c8155eb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f05c815c970 0x7f05c81f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): picked mon.noname-b con 0x7f05c815c970 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f05c815d340 con 0x7f05c8155eb0 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f05c815d1c0 con 0x7f05c815c970 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f05c815d4c0 con 0x7f05c806ff20 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05cd224640 10 monclient(hunting): authenticate will time out at 1085.672680s 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.945+0000 7f05c6575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c8155eb0 0x7f05c8152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f05c815c970 0x7f05c81f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f05c806ff20 0x7f05c8147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c8155eb0 0x7f05c8152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40060/0 (socket says 10.20.193.6:40060) 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 1 -- 10.20.193.6:0/4093485661 learned_addr learned my addr 10.20.193.6:0/4093485661 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 10 monclient(hunting): get_auth_request con 0x7f05c8155eb0 auth_method 0 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 10 monclient(hunting): get_auth_request con 0x7f05c806ff20 auth_method 0 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 10 monclient(hunting): get_auth_request con 0x7f05c815c970 auth_method 0 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.946+0000 7f05c7577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 10 monclient(hunting): handle_auth_done global_id 4557 payload 824 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 1 -- 10.20.193.6:0/4093485661 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f05c806ff20 msgr2=0x7f05c8147f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f05c806ff20 0x7f05c8147f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 1 -- 10.20.193.6:0/4093485661 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f05c815c970 msgr2=0x7f05c81f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f05c815c970 0x7f05c81f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 1 monclient: found mon.noname-a 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.947+0000 7f05c6575640 1 -- 10.20.193.6:0/4093485661 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c81551e0 con 0x7f05c8155eb0 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.948+0000 7f05c6575640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c8155eb0 0x7f05c8152370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c480 tx=0x7f05b000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.948+0000 7f05a77fe640 10 client.?.objecter ms_handle_connect 0x7f05c8155eb0 2026-02-22T23:23:30.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.948+0000 7f05a77fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 1 -- 10.20.193.6:0/4093485661 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0017070 con 0x7f05c8155eb0 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: dump: 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:30.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 1 -- 10.20.193.6:0/4093485661 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b00052f0 con 0x7f05c8155eb0 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 5 monclient: authenticate success, global_id 4557 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 monclient: _renew_subs 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c81f6d80 con 0x7f05c8155eb0 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 client.4557.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 monclient: _renew_subs 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81f7340 con 0x7f05c8155eb0 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 1 librados: init done 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 1 -- 10.20.193.6:0/4093485661 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00036f0 con 0x7f05c8155eb0 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05c5573640 4 set_mon_vals no callback set 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05a77fe640 10 monclient: dump: 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.956 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.950+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c8070300 con 0x7f05c8155eb0 2026-02-22T23:23:30.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.961+0000 7f05a77fe640 1 -- 10.20.193.6:0/4093485661 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f05b0003890 con 0x7f05c8155eb0 2026-02-22T23:23:30.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.961+0000 7f05a77fe640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0594078e50 0x7f059407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.962+0000 7f05c6d76640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0594078e50 0x7f059407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.962+0000 7f05c6d76640 10 monclient: get_auth_request con 0x7f0594078e50 auth_method 0 2026-02-22T23:23:30.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.962+0000 7f05a77fe640 10 client.4557.objecter ms_handle_connect 0x7f0594078e50 2026-02-22T23:23:30.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.962+0000 7f05c6d76640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0594078e50 0x7f059407b310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05bc039000 tx=0x7f05bc033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.964 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2026-02-22T23:23:30.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.963+0000 7f05a77fe640 1 -- 10.20.193.6:0/4093485661 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f05b005d390 con 0x7f05c8155eb0 2026-02-22T23:23:30.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.963+0000 7f05a77fe640 10 client.4557.objecter ms_dispatch2 0x7f05c8000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:30.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.963+0000 7f05a77fe640 3 client.4557.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:30.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.963+0000 7f05a77fe640 3 client.4557.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:30.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.964+0000 7f05a77fe640 20 client.4557.objecter dump_active .. 0 homeless 2026-02-22T23:23:30.985 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.984+0000 7fd0fad1b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:30.985 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.984+0000 7fd0fad1b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:30.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.984+0000 7fd0fad1b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:30.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.984+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd0f40c77d0 con 0x7fd0f41527d0 2026-02-22T23:23:30.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.987+0000 7fa1c6061640 10 monclient: get_monmap_and_config 2026-02-22T23:23:30.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.987+0000 7fa1c6061640 10 monclient: build_initial_monmap 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.987+0000 7fa1c6061640 10 monclient: monmap: 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:30.988228+0000 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:30.988228+0000 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:30.989 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 1 Processor -- start 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 1 -- start start 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient: init 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient: _add_conns ranks=[4,5,2,3,1,0] 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa1c006ff20 0x7fa1c0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient(hunting): picked mon.noname-b con 0x7fa1c006ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa1c015d5e0 0x7fa1c015da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient(hunting): picked mon.noname-c con 0x7fa1c015d5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa1c014ff60 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c0159ae0 con 0x7fa1c014ff60 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c0159da0 con 0x7fa1c006ff20 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c015a080 con 0x7fa1c015d5e0 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.988+0000 7fa1c6061640 10 monclient(hunting): authenticate will time out at 1085.715717s 2026-02-22T23:23:30.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa1c006ff20 0x7fa1c0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1beffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa1c015d5e0 0x7fa1c015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa1c006ff20 0x7fa1c0070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36770/0 (socket says 10.20.193.6:36770) 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 1 -- 10.20.193.6:0/323776172 learned_addr learned my addr 10.20.193.6:0/323776172 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 10 monclient(hunting): get_auth_request con 0x7fa1c006ff20 auth_method 0 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1beffd640 10 monclient(hunting): get_auth_request con 0x7fa1c015d5e0 auth_method 0 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1beffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1beffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.989+0000 7fa1bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 10 monclient(hunting): handle_auth_done global_id 4561 payload 274 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 1 -- 10.20.193.6:0/323776172 >> v2:10.20.193.80:3300/0 conn(0x7fa1c015d5e0 msgr2=0x7fa1c015da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 1 --2- 10.20.193.6:0/323776172 >> v2:10.20.193.80:3300/0 conn(0x7fa1c015d5e0 0x7fa1c015da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 1 -- 10.20.193.6:0/323776172 >> v1:10.20.193.80:6789/0 conn(0x7fa1c014ff60 legacy=0x7fa1c0150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:30.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 1 monclient: found mon.noname-b 2026-02-22T23:23:30.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:30.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:30.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.990+0000 7fa1bf7fe640 1 -- 10.20.193.6:0/323776172 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c015a360 con 0x7fa1c006ff20 2026-02-22T23:23:30.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.992+0000 7fa1bf7fe640 1 --2- 10.20.193.6:0/323776172 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c006ff20 0x7fa1c0070300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00708f0 tx=0x7fa1ac02ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5fd838b559c3749 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:30.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 1 -- 10.20.193.6:0/323776172 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac03c070 con 0x7fa1c006ff20 2026-02-22T23:23:30.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:30.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: dump: 2026-02-22T23:23:30.996 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.996 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.996 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 1 -- 10.20.193.6:0/323776172 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ac02fbd0 con 0x7fa1c006ff20 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1c6061640 5 monclient: authenticate success, global_id 4561 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1c6061640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:30.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 1 -- 10.20.193.6:0/323776172 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac037440 con 0x7fa1c006ff20 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1be7fc640 10 monclient: dump: 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1c6061640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.994+0000 7fa1c6061640 4 set_mon_vals no callback set 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 10 monclient: shutdown 2026-02-22T23:23:30.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 -- 10.20.193.6:0/323776172 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c006ff20 msgr2=0x7fa1c0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 --2- 10.20.193.6:0/323776172 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c006ff20 0x7fa1c0070300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00708f0 tx=0x7fa1ac02ef60 comp rx=0 tx=0).stop 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 -- 10.20.193.6:0/323776172 shutdown_connections 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 --2- 10.20.193.6:0/323776172 >> v2:10.20.193.80:3300/0 conn(0x7fa1c015d5e0 0x7fa1c015da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 --2- 10.20.193.6:0/323776172 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c006ff20 0x7fa1c0070300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 -- 10.20.193.6:0/323776172 >> 10.20.193.6:0/323776172 conn(0x7fa1c00cd780 msgr2=0x7fa1c00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 -- 10.20.193.6:0/323776172 shutdown_connections 2026-02-22T23:23:30.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 1 -- 10.20.193.6:0/323776172 wait complete. 2026-02-22T23:23:31.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.995+0000 7fa1c6061640 10 monclient: monmap: 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 librados: starting msgr at 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 librados: starting objecter 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 Processor -- start 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 -- start start 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 librados: setting wanted keys 2026-02-22T23:23:31.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 librados: calling monclient init 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient: init 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1c006ff20 0x7fa1c006ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): picked mon.noname-c con 0x7fa1c006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c014ff60 0x7fa1c014e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): picked mon.noname-b con 0x7fa1c014ff60 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1c015d5e0 0x7fa1c014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): picked mon.noname-a con 0x7fa1c015d5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c00c95a0 con 0x7fa1c015d5e0 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c015a920 con 0x7fa1c014ff60 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c00c69f0 con 0x7fa1c006ff20 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.996+0000 7fa1c6061640 10 monclient(hunting): authenticate will time out at 1085.723780s 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c014ff60 0x7fa1c014e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1bf7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1c006ff20 0x7fa1c006ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c014ff60 0x7fa1c014e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36780/0 (socket says 10.20.193.6:36780) 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 1 -- 10.20.193.6:0/3117020877 learned_addr learned my addr 10.20.193.6:0/3117020877 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 10 monclient(hunting): get_auth_request con 0x7fa1c014ff60 auth_method 0 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1bf7fe640 10 monclient(hunting): get_auth_request con 0x7fa1c006ff20 auth_method 0 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1bf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1bf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1bf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.997+0000 7fa1beffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1bf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1bf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1bf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 10 monclient(hunting): handle_auth_done global_id 4564 payload 824 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 1 -- 10.20.193.6:0/3117020877 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1c006ff20 msgr2=0x7fa1c006ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1c006ff20 0x7fa1c006ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 1 -- 10.20.193.6:0/3117020877 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1c015d5e0 msgr2=0x7fa1c014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1c015d5e0 0x7fa1c014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:30.998+0000 7fa1beffd640 1 -- 10.20.193.6:0/3117020877 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c006d6b0 con 0x7fa1c014ff60 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.000+0000 7fa1beffd640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c014ff60 0x7fa1c014e200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000e890 tx=0x7fa1b000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.000+0000 7fa19bfff640 10 client.?.objecter ms_handle_connect 0x7fa1c014ff60 2026-02-22T23:23:31.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.000+0000 7fa19bfff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa19bfff640 1 -- 10.20.193.6:0/3117020877 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0017070 con 0x7fa1c014ff60 2026-02-22T23:23:31.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa19bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa19bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa19bfff640 10 monclient: dump: 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa19bfff640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa19bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa19bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 5 monclient: authenticate success, global_id 4564 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 monclient: _renew_subs 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c006d990 con 0x7fa1c014ff60 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 client.4564.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 monclient: _renew_subs 2026-02-22T23:23:31.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c006df50 con 0x7fa1c014ff60 2026-02-22T23:23:31.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 1 librados: init done 2026-02-22T23:23:31.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.001+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c0070300 con 0x7fa1c014ff60 2026-02-22T23:23:31.012 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-22T23:23:31.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.030+0000 7f9d62f44640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.030+0000 7f9d62f44640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.030+0000 7f9d62f44640 10 monclient: monmap: 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.031085+0000 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.031085+0000 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.032 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.030+0000 7f9d62f44640 1 Processor -- start 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 1 -- start start 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient: init 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient: _add_conns ranks=[1,4,0,5,3,2] 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f9d5c1527d0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f9d5c152bb0 0x7f9d5c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient(hunting): picked mon.noname-b con 0x7f9d5c152bb0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9d5c0c5fc0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c06e300 con 0x7f9d5c0c5fc0 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d5c06e5e0 con 0x7f9d5c1527d0 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5c06f1f0 con 0x7f9d5c152bb0 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.031+0000 7f9d62f44640 10 monclient(hunting): authenticate will time out at 1085.758881s 2026-02-22T23:23:31.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 1 -- 10.20.193.6:0/1201365241 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7403c070 con 0x7f5b8c0c99a0 2026-02-22T23:23:31.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:31.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: dump: 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.064 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 1 -- 10.20.193.6:0/1201365241 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7402fba0 con 0x7f5b8c0c99a0 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 1 -- 10.20.193.6:0/1201365241 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b74037440 con 0x7f5b8c0c99a0 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b89573640 10 monclient: dump: 2026-02-22T23:23:31.065 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b90e2b640 5 monclient: authenticate success, global_id 4613 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b90e2b640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.060+0000 7f5b90e2b640 4 set_mon_vals no callback set 2026-02-22T23:23:31.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 10 monclient: shutdown 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 1 -- 10.20.193.6:0/1201365241 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c0c99a0 msgr2=0x7f5b8c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/1201365241 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c0c99a0 0x7f5b8c070300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5b740098d0 tx=0x7f5b7402eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 1 -- 10.20.193.6:0/1201365241 shutdown_connections 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/1201365241 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c0c99a0 0x7f5b8c070300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 1 -- 10.20.193.6:0/1201365241 >> 10.20.193.6:0/1201365241 conn(0x7f5b8c0cd780 msgr2=0x7f5b8c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 1 -- 10.20.193.6:0/1201365241 shutdown_connections 2026-02-22T23:23:31.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 1 -- 10.20.193.6:0/1201365241 wait complete. 2026-02-22T23:23:31.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f05a77fe640 1 -- 10.20.193.6:0/4093485661 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f05b0021020 con 0x7f05c8155eb0 2026-02-22T23:23:31.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f05a77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f05a77fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.061+0000 7f5b90e2b640 10 monclient: monmap: 2026-02-22T23:23:31.070 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.070 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 librados: starting msgr at 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 librados: starting objecter 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 Processor -- start 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 -- start start 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 librados: setting wanted keys 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 librados: calling monclient init 2026-02-22T23:23:31.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient: init 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c06ff20 0x7f5b8c1f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): picked mon.noname-c con 0x7f5b8c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5b8c0c99a0 0x7f5b8c06e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): picked mon.noname-b con 0x7f5b8c0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5b8c15a5e0 0x7f5b8c06e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): picked mon.noname-a con 0x7f5b8c15a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c150520 con 0x7f5b8c15a5e0 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0c8a10 con 0x7f5b8c0c99a0 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1506a0 con 0x7f5b8c06ff20 2026-02-22T23:23:31.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.062+0000 7f5b90e2b640 10 monclient(hunting): authenticate will time out at 1085.789899s 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c06ff20 0x7f5b8c1f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c06ff20 0x7f5b8c1f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58886/0 (socket says 10.20.193.6:58886) 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 1 -- 10.20.193.6:0/3196546808 learned_addr learned my addr 10.20.193.6:0/3196546808 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 10 monclient(hunting): get_auth_request con 0x7f5b8c06ff20 auth_method 0 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f5b8a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f9d614ba640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f9d5c0c5fc0 0x7f9d5c0c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35566/0 (socket says 10.20.193.6:35566) 2026-02-22T23:23:31.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.063+0000 7f9d614ba640 1 -- 10.20.193.6:0/465241082 learned_addr learned my addr 10.20.193.6:0/465241082 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:31.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 10 monclient(hunting): handle_auth_done global_id 4625 payload 824 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 1 -- 10.20.193.6:0/3196546808 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5b8c0c99a0 msgr2=0x7f5b8c06e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5b8c0c99a0 0x7f5b8c06e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 1 -- 10.20.193.6:0/3196546808 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5b8c15a5e0 msgr2=0x7f5b8c06e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5b8c15a5e0 0x7f5b8c06e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 1 monclient: found mon.noname-c 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.065+0000 7f5b8a575640 1 -- 10.20.193.6:0/3196546808 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c155d90 con 0x7f5b8c06ff20 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.067+0000 7f5b8a575640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c06ff20 0x7f5b8c1f51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c730 tx=0x7f5b8000cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.067+0000 7f5b72ffd640 10 client.?.objecter ms_handle_connect 0x7f5b8c06ff20 2026-02-22T23:23:31.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.067+0000 7f5b72ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.067+0000 7fd0e97fa640 1 -- 10.20.193.6:0/844521614 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0dc065f60 con 0x7fd0f41527d0 2026-02-22T23:23:31.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.067+0000 7fd0e97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 8}] 2026-02-22T23:23:31.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.067+0000 7fd0e97fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.076 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:31.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.068+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 648982117 0 0) 0x7f9d5c06e300 con 0x7f9d5c0c5fc0 2026-02-22T23:23:31.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.068+0000 7f9d537fe640 10 monclient(hunting): _negotiate 2026-02-22T23:23:31.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.068+0000 7f9d537fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.068+0000 7f9d537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.068+0000 7f9d537fe640 10 monclient(hunting): my global_id is 4560 2026-02-22T23:23:31.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.068+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d48003620 con 0x7f9d5c0c5fc0 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 1 -- 10.20.193.6:0/3196546808 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b800052d0 con 0x7f5b8c06ff20 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: dump: 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.078 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 1 -- 10.20.193.6:0/3196546808 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b80005470 con 0x7f5b8c06ff20 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 1 -- 10.20.193.6:0/3196546808 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b800038d0 con 0x7f5b8c06ff20 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b88d72640 4 set_mon_vals no callback set 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b72ffd640 10 monclient: dump: 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.079 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 5 monclient: authenticate success, global_id 4625 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 monclient: _renew_subs 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c156070 con 0x7f5b8c06ff20 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 client.4625.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 monclient: _renew_subs 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c156630 con 0x7f5b8c06ff20 2026-02-22T23:23:31.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 1 librados: init done 2026-02-22T23:23:31.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 10 librados: watch_flush enter 2026-02-22T23:23:31.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 10 librados: watch_flush exit 2026-02-22T23:23:31.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 20 client.4548.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 10 client.4548.objecter successfully canceled tick 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd0c4078e30 msgr2=0x7fd0c407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd0c4078e30 0x7fd0c407b2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4039000 tx=0x7fd0e4033000 comp rx=0 tx=0).stop 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 10 monclient: shutdown 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 msgr2=0x7fd0f4157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f4157150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00e8c0 tx=0x7fd0dc00ed90 comp rx=0 tx=0).stop 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 shutdown_connections 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd0c4078e30 0x7fd0c407b2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd0f4152bb0 0x7fd0f41f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd0f41527d0 0x7fd0f4157150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 --2- 10.20.193.6:0/844521614 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd0f40c5fc0 0x7fd0f41f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 >> 10.20.193.6:0/844521614 conn(0x7fd0f40cd780 msgr2=0x7fd0f40c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 shutdown_connections 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 -- 10.20.193.6:0/844521614 wait complete. 2026-02-22T23:23:31.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7fd0fad1b640 1 librados: shutdown 2026-02-22T23:23:31.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.069+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c070300 con 0x7f5b8c06ff20 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 760837496 0 0) 0x7f9d48003620 con 0x7f9d5c0c5fc0 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 >> v2:10.20.193.40:3300/0 conn(0x7f9d5c152bb0 msgr2=0x7f9d5c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 --2- 10.20.193.6:0/465241082 >> v2:10.20.193.40:3300/0 conn(0x7f9d5c152bb0 0x7f9d5c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 >> v1:10.20.193.40:6789/0 conn(0x7f9d5c1527d0 legacy=0x7f9d5c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c06e8c0 con 0x7f9d5c0c5fc0 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f9d58003280 con 0x7f9d5c0c5fc0 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d62f44640 5 monclient: authenticate success, global_id 4560 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d62f44640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: dump: 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.085 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 >> v1:10.20.193.6:6789/0 conn(0x7f9d5c0c5fc0 legacy=0x7f9d5c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9d48003ba0 0x7f9d48003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): picked mon.c con 0x7f9d48003ba0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d48004900 0x7f9d48004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): picked mon.a con 0x7f9d48004900 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9d48005510 0x7f9d4800de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): picked mon.b con 0x7f9d48005510 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c06f1f0 con 0x7f9d48004900 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4800e6e0 con 0x7f9d48005510 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4800e8e0 con 0x7f9d48003ba0 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d537fe640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.073+0000 7f9d62f44640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.074+0000 7f9d53fff640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d48004900 0x7f9d48004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.075+0000 7f9d53fff640 10 monclient(hunting): get_auth_request con 0x7f9d48004900 auth_method 0 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.075+0000 7f9d53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.075+0000 7f9d53fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.075+0000 7f9d53fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.076+0000 7f9d53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.076+0000 7f9d53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.076+0000 7f9d53fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:31.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 10 monclient(hunting): handle_auth_done global_id 4560 payload 293 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 1 -- 10.20.193.6:0/465241082 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9d48003ba0 msgr2=0x7f9d48003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9d48003ba0 0x7f9d48003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 1 -- 10.20.193.6:0/465241082 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9d48005510 msgr2=0x7f9d4800de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9d48005510 0x7f9d4800de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 1 monclient: found mon.a 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d614ba640 1 -- 10.20.193.6:0/465241082 reap_dead start 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 1 -- 10.20.193.6:0/465241082 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d48003620 con 0x7f9d48004900 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.077+0000 7f9d53fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.078+0000 7f1da20a9640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.078+0000 7f1da20a9640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.078+0000 7f1da20a9640 10 monclient: monmap: 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.078946+0000 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.078946+0000 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.087 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.8 2026-02-22T23:23:31.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.078+0000 7f9d53fff640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d48004900 0x7f9d48004d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009980 tx=0x7f9d4402ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecd1579b94907f server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.088 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.078+0000 7f1da20a9640 1 Processor -- start 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.078+0000 7f1da20a9640 1 -- start start 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.078+0000 7f1da20a9640 10 monclient: init 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient: _add_conns ranks=[2,1,5,4,3,0] 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1d9c06ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1d9c0c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f1d9c15a5e0 0x7f1d9c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient(hunting): picked mon.noname-c con 0x7f1d9c15a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d9c0c7e90 con 0x7f1d9c0c99a0 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d9c0c8170 con 0x7f1d9c06ff20 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1d9c159a40 con 0x7f1d9c15a5e0 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.079+0000 7f1da20a9640 10 monclient(hunting): authenticate will time out at 1085.806324s 2026-02-22T23:23:31.090 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4403c070 con 0x7f9d48004900 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 10 monclient: dump: 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.091 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4402f7b0 con 0x7f9d48004900 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 1 -- 10.20.193.6:0/465241082 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44009550 con 0x7f9d48004900 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 4 set_mon_vals no callback set 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d537fe640 10 monclient: dump: 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 10 monclient: shutdown 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 1 -- 10.20.193.6:0/465241082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d48004900 msgr2=0x7f9d48004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d48004900 0x7f9d48004d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009980 tx=0x7f9d4402ed50 comp rx=0 tx=0).stop 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 1 -- 10.20.193.6:0/465241082 shutdown_connections 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 1 --2- 10.20.193.6:0/465241082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d48004900 0x7f9d48004d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 1 -- 10.20.193.6:0/465241082 >> 10.20.193.6:0/465241082 conn(0x7f9d5c0cd780 msgr2=0x7f9d5c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 1 -- 10.20.193.6:0/465241082 shutdown_connections 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.080+0000 7f9d62f44640 1 -- 10.20.193.6:0/465241082 wait complete. 2026-02-22T23:23:31.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 10 monclient: monmap: 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 1 librados: starting msgr at 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 1 librados: starting objecter 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 1 Processor -- start 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 1 -- start start 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 1 librados: setting wanted keys 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 1 librados: calling monclient init 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 10 monclient: init 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.081+0000 7f9d62f44640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9d48004900 0x7f9d5c147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): picked mon.noname-c con 0x7f9d48004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d5c1f7bb0 0x7f9d5c157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): picked mon.noname-a con 0x7f9d5c1f7bb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9d5c158000 0x7f9d5c0bc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): picked mon.noname-b con 0x7f9d5c158000 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c06ee80 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c0c6470 con 0x7f9d5c158000 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5c0c5a80 con 0x7f9d48004900 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d62f44640 10 monclient(hunting): authenticate will time out at 1085.809029s 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d5c1f7bb0 0x7f9d5c157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d5c1f7bb0 0x7f9d5c157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40108/0 (socket says 10.20.193.6:40108) 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 1 -- 10.20.193.6:0/2473396999 learned_addr learned my addr 10.20.193.6:0/2473396999 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 10 monclient(hunting): get_auth_request con 0x7f9d5c1f7bb0 auth_method 0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.082+0000 7f9d53fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 10 monclient(hunting): handle_auth_done global_id 4563 payload 824 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 1 -- 10.20.193.6:0/2473396999 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9d48004900 msgr2=0x7f9d5c147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9d48004900 0x7f9d5c147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 1 -- 10.20.193.6:0/2473396999 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9d5c158000 msgr2=0x7f9d5c0bc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9d5c158000 0x7f9d5c0bc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 1 monclient: found mon.noname-a 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.084+0000 7f9d53fff640 1 -- 10.20.193.6:0/2473396999 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5c1f6d90 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.085+0000 7f9d53fff640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d5c1f7bb0 0x7f9d5c157ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d44002410 tx=0x7f9d440048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.085+0000 7f9d517fa640 10 client.?.objecter ms_handle_connect 0x7f9d5c1f7bb0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.085+0000 7f9d517fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 1 -- 10.20.193.6:0/2473396999 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4403c070 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: dump: 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.094 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 1 -- 10.20.193.6:0/2473396999 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d44046070 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 1 -- 10.20.193.6:0/2473396999 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44006e60 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d52ffd640 4 set_mon_vals no callback set 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d517fa640 10 monclient: dump: 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 5 monclient: authenticate success, global_id 4563 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 10 monclient: _renew_subs 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5c156740 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 10 client.4563.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 10 monclient: _renew_subs 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5c156d00 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.087+0000 7f9d62f44640 1 librados: init done 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.088+0000 7f9d62f44640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.088+0000 7f9d62f44640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.088+0000 7f9d62f44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.088+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d5c06da60 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.096 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2026-02-22T23:23:31.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.100+0000 7f9d517fa640 1 -- 10.20.193.6:0/2473396999 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9d44002b10 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.100+0000 7f9d517fa640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9d2c078e30 0x7f9d2c07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.101+0000 7f9d60cb9640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9d2c078e30 0x7f9d2c07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.101+0000 7f9d60cb9640 10 monclient: get_auth_request con 0x7f9d2c078e30 auth_method 0 2026-02-22T23:23:31.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.101+0000 7f9d517fa640 10 client.4563.objecter ms_handle_connect 0x7f9d2c078e30 2026-02-22T23:23:31.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.101+0000 7f9d60cb9640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9d2c078e30 0x7f9d2c07b2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0097d0 tx=0x7f9d4c006db0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.102+0000 7f9d517fa640 1 -- 10.20.193.6:0/2473396999 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f9d44082b20 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.102+0000 7f9d517fa640 10 client.4563.objecter ms_dispatch2 0x7f9d5c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.102+0000 7f9d517fa640 3 client.4563.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.102+0000 7f9d517fa640 3 client.4563.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.102+0000 7f9d517fa640 20 client.4563.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.108+0000 7fa19bfff640 1 -- 10.20.193.6:0/3117020877 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b0005a90 con 0x7fa1c014ff60 2026-02-22T23:23:31.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.108+0000 7fa19bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.108+0000 7fa19bfff640 1 -- 10.20.193.6:0/3117020877 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0021060 con 0x7fa1c014ff60 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.108+0000 7fa19bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.108+0000 7fa19bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.108+0000 7fa19bfff640 10 monclient: dump: 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.108+0000 7fa1bdffb640 4 set_mon_vals no callback set 2026-02-22T23:23:31.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.109+0000 7fa19bfff640 1 -- 10.20.193.6:0/3117020877 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa1b000c940 con 0x7fa1c014ff60 2026-02-22T23:23:31.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.110+0000 7fa19bfff640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa190078e30 0x7fa19007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.110+0000 7fa1bf7fe640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa190078e30 0x7fa19007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.110+0000 7fa1bf7fe640 10 monclient: get_auth_request con 0x7fa190078e30 auth_method 0 2026-02-22T23:23:31.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.111+0000 7fa1bf7fe640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa190078e30 0x7fa19007b2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1c015af60 tx=0x7fa1ac033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.111+0000 7fa19bfff640 10 client.4564.objecter ms_handle_connect 0x7fa190078e30 2026-02-22T23:23:31.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.111+0000 7fa19bfff640 1 -- 10.20.193.6:0/3117020877 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fa1b005e380 con 0x7fa1c014ff60 2026-02-22T23:23:31.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.111+0000 7fa19bfff640 10 client.4564.objecter ms_dispatch2 0x7fa1c0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.111+0000 7fa19bfff640 3 client.4564.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.111+0000 7fa19bfff640 3 client.4564.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.111+0000 7fa19bfff640 20 client.4564.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.118+0000 7fa19bfff640 1 -- 10.20.193.6:0/3117020877 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa1b0062340 con 0x7fa1c014ff60 2026-02-22T23:23:31.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.118+0000 7fa19bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.118+0000 7fa19bfff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.121+0000 7f1d9affd640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f1d9c0c99a0 0x7f1d9c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60412/0 (socket says 10.20.193.6:60412) 2026-02-22T23:23:31.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.121+0000 7f1d9affd640 1 -- 10.20.193.6:0/2853106547 learned_addr learned my addr 10.20.193.6:0/2853106547 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:31.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.124+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075808759 0 0) 0x7f1d9c0c7e90 con 0x7f1d9c0c99a0 2026-02-22T23:23:31.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.124+0000 7f1d9a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:23:31.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.124+0000 7f1d9a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.124+0000 7f1d9a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.124+0000 7f1d9a7fc640 10 monclient(hunting): my global_id is 4567 2026-02-22T23:23:31.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.124+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d80003620 con 0x7f1d9c0c99a0 2026-02-22T23:23:31.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1701251549 0 0) 0x7f1d80003620 con 0x7f1d9c0c99a0 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 >> v2:10.20.193.80:3300/0 conn(0x7f1d9c15a5e0 msgr2=0x7f1d9c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 --2- 10.20.193.6:0/2853106547 >> v2:10.20.193.80:3300/0 conn(0x7f1d9c15a5e0 0x7f1d9c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 >> v1:10.20.193.80:6789/0 conn(0x7f1d9c06ff20 legacy=0x7f1d9c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c0c8450 con 0x7f1d9c0c99a0 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f1d8c002f00 con 0x7f1d9c0c99a0 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1da20a9640 5 monclient: authenticate success, global_id 4567 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1da20a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: dump: 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 >> v1:10.20.193.40:6789/0 conn(0x7f1d9c0c99a0 legacy=0x7f1d9c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:31.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d80003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): picked mon.b con 0x7f1d80003ba0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d80004900 0x7f1d80004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): picked mon.a con 0x7f1d80004900 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d80005510 0x7f1d8000bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): picked mon.c con 0x7f1d80005510 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c159a40 con 0x7f1d80004900 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8000c450 con 0x7f1d80003ba0 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8000c650 con 0x7f1d80005510 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1d9a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.126+0000 7f1da20a9640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d80003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 10 monclient(hunting): get_auth_request con 0x7f1d80003ba0 auth_method 0 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.127+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_done global_id 4567 payload 293 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 1 -- 10.20.193.6:0/2853106547 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d80005510 msgr2=0x7f1d8000bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d80005510 0x7f1d8000bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 1 -- 10.20.193.6:0/2853106547 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d80004900 msgr2=0x7f1d80004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d80004900 0x7f1d80004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 1 monclient: found mon.b 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 1 -- 10.20.193.6:0/2853106547 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d80003620 con 0x7f1d80003ba0 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9bfff640 1 -- 10.20.193.6:0/2853106547 reap_dead start 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.128+0000 7f1d9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.129+0000 7f1d9b7fe640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d80003fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1d88002f60 tx=0x7f1d8802eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7602d8fa91dca934 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88002b30 con 0x7f1d80003ba0 2026-02-22T23:23:31.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 10 monclient: dump: 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d88005290 con 0x7f1d80003ba0 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1da20a9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1da20a9640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 1 -- 10.20.193.6:0/2853106547 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88003710 con 0x7f1d80003ba0 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1da20a9640 4 set_mon_vals no callback set 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.131+0000 7f1d9a7fc640 10 monclient: dump: 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.133 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 10 monclient: shutdown 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 -- 10.20.193.6:0/2853106547 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 msgr2=0x7f1d80003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d80003fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1d88002f60 tx=0x7f1d8802eca0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 -- 10.20.193.6:0/2853106547 shutdown_connections 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 --2- 10.20.193.6:0/2853106547 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d80003fa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 -- 10.20.193.6:0/2853106547 >> 10.20.193.6:0/2853106547 conn(0x7f1d9c0cd780 msgr2=0x7f1d9c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 -- 10.20.193.6:0/2853106547 shutdown_connections 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 -- 10.20.193.6:0/2853106547 wait complete. 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 10 monclient: monmap: 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.134 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 librados: starting msgr at 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.132+0000 7f1da20a9640 1 librados: starting objecter 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 Processor -- start 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 -- start start 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 librados: setting wanted keys 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 librados: calling monclient init 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient: init 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d9c148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): picked mon.noname-b con 0x7f1d80003ba0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d9c148a90 0x7f1d9c073600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): picked mon.noname-a con 0x7f1d9c148a90 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d9c073b40 0x7f1d9c1ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): picked mon.noname-c con 0x7f1d9c073b40 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c070300 con 0x7f1d9c148a90 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c0c8a10 con 0x7f1d80003ba0 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c070480 con 0x7f1d9c073b40 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1da20a9640 10 monclient(hunting): authenticate will time out at 1085.860553s 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1d9b7fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d9c148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1d9b7fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d9c148550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36838/0 (socket says 10.20.193.6:36838) 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1d9b7fe640 1 -- 10.20.193.6:0/363057409 learned_addr learned my addr 10.20.193.6:0/363057409 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1d9b7fe640 10 monclient(hunting): get_auth_request con 0x7f1d80003ba0 auth_method 0 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1d9b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1d9b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.133+0000 7f1d9b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.134+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.134+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.134+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 10 monclient(hunting): handle_auth_done global_id 4570 payload 824 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 1 -- 10.20.193.6:0/363057409 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d9c073b40 msgr2=0x7f1d9c1ecff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d9c073b40 0x7f1d9c1ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 1 -- 10.20.193.6:0/363057409 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d9c148a90 msgr2=0x7f1d9c073600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d9c148a90 0x7f1d9c073600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.135+0000 7f1d9b7fe640 1 -- 10.20.193.6:0/363057409 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c148f10 con 0x7f1d80003ba0 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.136+0000 7f1d9b7fe640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d9c148550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802ece0 tx=0x7f1d880027c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.136+0000 7f1d77fff640 10 client.?.objecter ms_handle_connect 0x7f1d80003ba0 2026-02-22T23:23:31.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.136+0000 7f1d77fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1d77fff640 1 -- 10.20.193.6:0/363057409 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88049070 con 0x7f1d80003ba0 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1d77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1d77fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1d77fff640 10 monclient: dump: 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1d77fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1d77fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1d77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 5 monclient: authenticate success, global_id 4570 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 10 monclient: _renew_subs 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c073fc0 con 0x7f1d80003ba0 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 10 client.4570.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 10 monclient: _renew_subs 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c06e540 con 0x7f1d80003ba0 2026-02-22T23:23:31.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.137+0000 7f1da20a9640 1 librados: init done 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1da20a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1da20a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1da20a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c159ed0 con 0x7f1d80003ba0 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1d77fff640 1 -- 10.20.193.6:0/363057409 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d88005290 con 0x7f1d80003ba0 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1d77fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1d77fff640 1 -- 10.20.193.6:0/363057409 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88006cb0 con 0x7f1d80003ba0 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1d77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1d77fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1d77fff640 10 monclient: dump: 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.138+0000 7f1d99ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:31.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.149+0000 7f05cd224640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:31.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.149+0000 7f05cd224640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:31.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.149+0000 7f05cd224640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.149+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f05c815c750 con 0x7f05c8155eb0 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.162+0000 7fa20f577640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.162+0000 7fa20f577640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.162+0000 7fa20f577640 10 monclient: monmap: 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.163468+0000 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.163468+0000 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.164 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 1 Processor -- start 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 1 -- start start 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient: init 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient: _add_conns ranks=[0,3,5,4,2,1] 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fa210155a50 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa210072e50 0x7fa2100732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient(hunting): picked mon.noname-a con 0x7fa210072e50 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa2100c9160 0x7fa2100c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient(hunting): picked mon.noname-c con 0x7fa2100c9160 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa21015fa90 con 0x7fa210155a50 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa21015f860 con 0x7fa210072e50 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa210160050 con 0x7fa2100c9160 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.163+0000 7fa20f577640 10 monclient(hunting): authenticate will time out at 1085.890854s 2026-02-22T23:23:31.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.166+0000 7f8fe7bd3640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.166+0000 7f8fe7bd3640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.166+0000 7f8fe7bd3640 10 monclient: monmap: 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.167434+0000 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.167434+0000 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.168 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 1 Processor -- start 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 1 -- start start 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient: init 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient: _add_conns ranks=[0,3,2,4,5,1] 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8fe006ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8fe0155eb0 0x7f8fe0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient(hunting): picked mon.noname-a con 0x7f8fe0155eb0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8fe015c970 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe00725b0 con 0x7f8fe006ff20 2026-02-22T23:23:31.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe0072890 con 0x7f8fe015c970 2026-02-22T23:23:31.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe0072b70 con 0x7f8fe0155eb0 2026-02-22T23:23:31.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.167+0000 7f8fe7bd3640 10 monclient(hunting): authenticate will time out at 1085.894790s 2026-02-22T23:23:31.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.171+0000 7f5b72ffd640 1 -- 10.20.193.6:0/3196546808 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5b80003a70 con 0x7f5b8c06ff20 2026-02-22T23:23:31.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.171+0000 7f5b72ffd640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b58078e50 0x7f5b5807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.173 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.172+0000 7f5b89d74640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b58078e50 0x7f5b5807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.173 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.172+0000 7f5b89d74640 10 monclient: get_auth_request con 0x7f5b58078e50 auth_method 0 2026-02-22T23:23:31.173 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.172+0000 7f5b89d74640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b58078e50 0x7f5b5807b310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b74039000 tx=0x7f5b74031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.173 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.172+0000 7f5b72ffd640 10 client.4625.objecter ms_handle_connect 0x7f5b58078e50 2026-02-22T23:23:31.174 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-22T23:23:31.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.173+0000 7f5b72ffd640 1 -- 10.20.193.6:0/3196546808 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f5b8005e120 con 0x7f5b8c06ff20 2026-02-22T23:23:31.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.173+0000 7f5b72ffd640 10 client.4625.objecter ms_dispatch2 0x7f5b8c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.173+0000 7f5b72ffd640 3 client.4625.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.173+0000 7f5b72ffd640 3 client.4625.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.173+0000 7f5b72ffd640 20 client.4625.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.180+0000 7f5b72ffd640 1 -- 10.20.193.6:0/3196546808 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5b800627e0 con 0x7f5b8c06ff20 2026-02-22T23:23:31.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.180+0000 7f5b72ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.180+0000 7f5b72ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.183+0000 7fa20ed76640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa2100c9160 0x7fa2100c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.183+0000 7fa20ed76640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa2100c9160 0x7fa2100c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58938/0 (socket says 10.20.193.6:58938) 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.183+0000 7fa20ed76640 1 -- 10.20.193.6:0/1155052744 learned_addr learned my addr 10.20.193.6:0/1155052744 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.183+0000 7fa20ed76640 10 monclient(hunting): get_auth_request con 0x7fa2100c9160 auth_method 0 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.183+0000 7fa20ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.183+0000 7fa20ed76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.183+0000 7fa20ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.184+0000 7fa20ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.184+0000 7fa20ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.184+0000 7fa20ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 10 monclient(hunting): handle_auth_done global_id 4628 payload 274 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 1 -- 10.20.193.6:0/1155052744 >> v2:10.20.193.6:3300/0 conn(0x7fa210072e50 msgr2=0x7fa2100732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 1 --2- 10.20.193.6:0/1155052744 >> v2:10.20.193.6:3300/0 conn(0x7fa210072e50 0x7fa2100732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 1 -- 10.20.193.6:0/1155052744 >> v1:10.20.193.6:6789/0 conn(0x7fa210155a50 legacy=0x7fa21015f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 1 monclient: found mon.noname-c 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.185+0000 7fa20ed76640 1 -- 10.20.193.6:0/1155052744 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21015fd70 con 0x7fa2100c9160 2026-02-22T23:23:31.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.187+0000 7fa20ed76640 1 --2- 10.20.193.6:0/1155052744 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa2100c9160 0x7fa2100c9590 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa21006f320 tx=0x7fa20002f270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58a676aacf121df server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.187+0000 7f8fe6149640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f8fe015c970 0x7f8fe00c7d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36304/0 (socket says 10.20.193.6:36304) 2026-02-22T23:23:31.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.187+0000 7f8fe6149640 1 -- 10.20.193.6:0/2918428150 learned_addr learned my addr 10.20.193.6:0/2918428150 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:31.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496271415 0 0) 0x7f8fe0072890 con 0x7f8fe015c970 2026-02-22T23:23:31.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7f8fe4946640 10 monclient(hunting): _negotiate 2026-02-22T23:23:31.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7f8fe4946640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7f8fe4946640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7f8fe4946640 10 monclient(hunting): my global_id is 4631 2026-02-22T23:23:31.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fc8003620 con 0x7f8fe015c970 2026-02-22T23:23:31.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 1 -- 10.20.193.6:0/1155052744 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20002fcd0 con 0x7fa2100c9160 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: dump: 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 1 -- 10.20.193.6:0/1155052744 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2000373f0 con 0x7fa2100c9160 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 1 -- 10.20.193.6:0/1155052744 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2000377c0 con 0x7fa2100c9160 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20d573640 10 monclient: dump: 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 5 monclient: authenticate success, global_id 4628 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 4 set_mon_vals no callback set 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 10 monclient: shutdown 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 -- 10.20.193.6:0/1155052744 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa2100c9160 msgr2=0x7fa2100c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 --2- 10.20.193.6:0/1155052744 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa2100c9160 0x7fa2100c9590 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa21006f320 tx=0x7fa20002f270 comp rx=0 tx=0).stop 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 -- 10.20.193.6:0/1155052744 shutdown_connections 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 --2- 10.20.193.6:0/1155052744 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa2100c9160 0x7fa2100c9590 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 --2- 10.20.193.6:0/1155052744 >> v2:10.20.193.6:3300/0 conn(0x7fa210072e50 0x7fa2100732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 -- 10.20.193.6:0/1155052744 >> 10.20.193.6:0/1155052744 conn(0x7fa2100cd480 msgr2=0x7fa2100cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 -- 10.20.193.6:0/1155052744 shutdown_connections 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.191+0000 7fa20f577640 1 -- 10.20.193.6:0/1155052744 wait complete. 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 10 monclient: monmap: 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.196 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 1 librados: starting msgr at 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 1 librados: starting objecter 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 1 Processor -- start 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 1 -- start start 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 1 librados: setting wanted keys 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 1 librados: calling monclient init 2026-02-22T23:23:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 10 monclient: init 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.192+0000 7fa20f577640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa210072e50 0x7fa2101f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): picked mon.noname-c con 0x7fa210072e50 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa2100c9160 0x7fa2101f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): picked mon.noname-a con 0x7fa2100c9160 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa210155a50 0x7fa2101f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): picked mon.noname-b con 0x7fa210155a50 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.198 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa210072930 con 0x7fa2100c9160 2026-02-22T23:23:31.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa210072650 con 0x7fa210155a50 2026-02-22T23:23:31.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100c9ad0 con 0x7fa210072e50 2026-02-22T23:23:31.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20f577640 10 monclient(hunting): authenticate will time out at 1085.920045s 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3714007589 0 0) 0x7f8fc8003620 con 0x7f8fe015c970 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 >> v2:10.20.193.6:3300/0 conn(0x7f8fe0155eb0 msgr2=0x7f8fe0070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 --2- 10.20.193.6:0/2918428150 >> v2:10.20.193.6:3300/0 conn(0x7f8fe0155eb0 0x7f8fe0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 >> v1:10.20.193.6:6789/0 conn(0x7f8fe006ff20 legacy=0x7f8fe01557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe0072e50 con 0x7f8fe015c970 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f8fdc003280 con 0x7f8fe015c970 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: dump: 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 >> v1:10.20.193.80:6789/0 conn(0x7f8fe015c970 legacy=0x7f8fe00c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8fc8003ba0 0x7f8fc8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): picked mon.b con 0x7f8fc8003ba0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fc8004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): picked mon.c con 0x7f8fc8004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fc8005510 0x7f8fc800bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): picked mon.a con 0x7f8fc8005510 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0072b70 con 0x7f8fc8005510 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800c450 con 0x7f8fc8003ba0 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800c650 con 0x7f8fc8004900 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7f8fe4946640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.193+0000 7fa20e575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa210072e50 0x7fa2101f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa210072e50 0x7fa2101f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:58950/0 (socket says 10.20.193.6:58950) 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 1 -- 10.20.193.6:0/2487973943 learned_addr learned my addr 10.20.193.6:0/2487973943 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 10 monclient(hunting): get_auth_request con 0x7fa210072e50 auth_method 0 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7f8fe5147640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fc8004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7fa20e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7f8fe5147640 10 monclient(hunting): get_auth_request con 0x7f8fc8004900 auth_method 0 2026-02-22T23:23:31.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7f8fe5147640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7f8fe5147640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.194+0000 7f8fe5147640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 10 monclient(hunting): handle_auth_done global_id 4634 payload 824 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 1 -- 10.20.193.6:0/2487973943 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa210155a50 msgr2=0x7fa2101f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa210155a50 0x7fa2101f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 1 -- 10.20.193.6:0/2487973943 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa2100c9160 msgr2=0x7fa2101f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa2100c9160 0x7fa2101f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 1 monclient: found mon.noname-c 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.196+0000 7fa20e575640 1 -- 10.20.193.6:0/2487973943 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2101f3fa0 con 0x7fa210072e50 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.197+0000 7f8fe5147640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.197+0000 7f8fe5147640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.197+0000 7f8fe5147640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 10 monclient(hunting): handle_auth_done global_id 4631 payload 293 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 1 -- 10.20.193.6:0/2918428150 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8fc8003ba0 msgr2=0x7f8fc8003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8fc8003ba0 0x7f8fc8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 1 -- 10.20.193.6:0/2918428150 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fc8005510 msgr2=0x7f8fc800bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fc8005510 0x7f8fc800bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 1 monclient: found mon.c 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 1 -- 10.20.193.6:0/2918428150 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc8003620 con 0x7f8fc8004900 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe5147640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe6149640 1 -- 10.20.193.6:0/2918428150 reap_dead start 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe7bd3640 5 monclient: authenticate success, global_id 4631 2026-02-22T23:23:31.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.198+0000 7f8fe7bd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.199+0000 7fa20e575640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa210072e50 0x7fa2101f7780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa20400b570 tx=0x7fa20400ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.199+0000 7fa1f6ffd640 10 client.?.objecter ms_handle_connect 0x7fa210072e50 2026-02-22T23:23:31.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.199+0000 7fa1f6ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.200+0000 7f8fe5147640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fc8004d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009980 tx=0x7f8fd402ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e632201b30eed710 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa1f6ffd640 1 -- 10.20.193.6:0/2487973943 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204013020 con 0x7fa210072e50 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa1f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa1f6ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa1f6ffd640 10 monclient: dump: 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.208 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa1f6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa1f6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa1f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 5 monclient: authenticate success, global_id 4634 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 10 monclient: _renew_subs 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2101f4280 con 0x7fa210072e50 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 10 client.4634.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 10 monclient: _renew_subs 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2101f6ca0 con 0x7fa210072e50 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.201+0000 7fa20f577640 1 librados: init done 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa20f577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa20f577640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa20f577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa210155e30 con 0x7fa210072e50 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa1f6ffd640 1 -- 10.20.193.6:0/2487973943 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa204004480 con 0x7fa210072e50 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa1f6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa1f6ffd640 1 -- 10.20.193.6:0/2487973943 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204009380 con 0x7fa210072e50 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa1f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa1f6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa20cd72640 4 set_mon_vals no callback set 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.202+0000 7fa1f6ffd640 10 monclient: dump: 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.209 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.210 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd403c070 con 0x7f8fc8004900 2026-02-22T23:23:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 10 monclient: dump: 2026-02-22T23:23:31.211 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd402f790 con 0x7f8fc8004900 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 1 -- 10.20.193.6:0/2918428150 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4009550 con 0x7f8fc8004900 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 4 set_mon_vals no callback set 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe4946640 10 monclient: dump: 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.212 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 10 monclient: shutdown 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2918428150 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 msgr2=0x7f8fc8004d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fc8004d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4009980 tx=0x7f8fd402ed30 comp rx=0 tx=0).stop 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2918428150 shutdown_connections 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2918428150 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fc8004d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2918428150 >> 10.20.193.6:0/2918428150 conn(0x7f8fe00cd780 msgr2=0x7f8fe00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2918428150 shutdown_connections 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.203+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2918428150 wait complete. 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient: monmap: 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 librados: starting msgr at 2026-02-22T23:23:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 librados: starting objecter 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 Processor -- start 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 -- start start 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 librados: setting wanted keys 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 librados: calling monclient init 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient: init 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fe0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): picked mon.noname-c con 0x7f8fc8004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8fe01e9970 0x7f8fe0153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): picked mon.noname-b con 0x7f8fe01e9970 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fe006da30 0x7f8fe006dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): picked mon.noname-a con 0x7f8fe006da30 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0070480 con 0x7f8fe006da30 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0070300 con 0x7f8fe01e9970 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0070600 con 0x7f8fc8004900 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.204+0000 7f8fe7bd3640 10 monclient(hunting): authenticate will time out at 1085.931939s 2026-02-22T23:23:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.206+0000 7f9d517fa640 1 -- 10.20.193.6:0/2473396999 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9d5c1f6d90 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.206+0000 7f9d517fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.206+0000 7f9d517fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.206+0000 7fa1c6061640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.206+0000 7fa1c6061640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.206+0000 7fa1c6061640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.206+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fa1c00c4a10 con 0x7fa1c014ff60 2026-02-22T23:23:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f05a77fe640 1 -- 10.20.193.6:0/4093485661 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f05b0061350 con 0x7f05c8155eb0 2026-02-22T23:23:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f05a77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2026-02-22T23:23:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f05a77fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.218 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f8fe6149640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fe006da30 0x7f8fe006dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f8fe6149640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fe006da30 0x7f8fe006dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40154/0 (socket says 10.20.193.6:40154) 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f8fe6149640 1 -- 10.20.193.6:0/2517309369 learned_addr learned my addr 10.20.193.6:0/2517309369 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f8fe6149640 10 monclient(hunting): get_auth_request con 0x7f8fe006da30 auth_method 0 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f8fe6149640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f8fe6149640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.217+0000 7f8fe6149640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.218+0000 7f8fe6149640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.218+0000 7f8fe6149640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.218+0000 7f8fe6149640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 10 librados: watch_flush enter 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 10 librados: watch_flush exit 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 20 client.4557.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 10 client.4557.objecter successfully canceled tick 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0594078e50 msgr2=0x7f059407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0594078e50 0x7f059407b310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05bc039000 tx=0x7f05bc033000 comp rx=0 tx=0).stop 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 10 monclient: shutdown 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c8155eb0 msgr2=0x7f05c8152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c8155eb0 0x7f05c8152370 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c480 tx=0x7f05b000c950 comp rx=0 tx=0).stop 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 shutdown_connections 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0594078e50 0x7f059407b310 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f05c815c970 0x7f05c81f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f05c8155eb0 0x7f05c8152370 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 --2- 10.20.193.6:0/4093485661 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f05c806ff20 0x7f05c8147f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 >> 10.20.193.6:0/4093485661 conn(0x7f05c80cd780 msgr2=0x7f05c8155570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 shutdown_connections 2026-02-22T23:23:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 -- 10.20.193.6:0/4093485661 wait complete. 2026-02-22T23:23:31.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.219+0000 7f05cd224640 1 librados: shutdown 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 10 monclient(hunting): handle_auth_done global_id 4566 payload 824 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 1 -- 10.20.193.6:0/2517309369 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 msgr2=0x7f8fe0154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fe0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 1 -- 10.20.193.6:0/2517309369 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8fe01e9970 msgr2=0x7f8fe0153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8fe01e9970 0x7f8fe0153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 1 monclient: found mon.noname-a 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.222+0000 7f8fe6149640 1 -- 10.20.193.6:0/2517309369 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe0152510 con 0x7f8fe006da30 2026-02-22T23:23:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.224+0000 7f8fe6149640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fe006da30 0x7f8fe006dee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00c4410 tx=0x7f8fdc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.224+0000 7f8fce7fc640 10 client.?.objecter ms_handle_connect 0x7f8fe006da30 2026-02-22T23:23:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.224+0000 7f8fce7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.4 2026-02-22T23:23:31.230 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 1 -- 10.20.193.6:0/2517309369 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00acc0 con 0x7f8fe006da30 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: dump: 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 1 -- 10.20.193.6:0/2517309369 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fdc00ae60 con 0x7f8fe006da30 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 1 -- 10.20.193.6:0/2517309369 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc014660 con 0x7f8fe006da30 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fce7fc640 10 monclient: dump: 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fcffff640 4 set_mon_vals no callback set 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 5 monclient: authenticate success, global_id 4566 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 monclient: _renew_subs 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe01527f0 con 0x7f8fe006da30 2026-02-22T23:23:31.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 client.4566.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 monclient: _renew_subs 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe006cf50 con 0x7f8fe006da30 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 1 librados: init done 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.229+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe0072510 con 0x7f8fe006da30 2026-02-22T23:23:31.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.242+0000 7f8fce7fc640 1 -- 10.20.193.6:0/2517309369 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8fdc01e420 con 0x7f8fe006da30 2026-02-22T23:23:31.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.242+0000 7f8fce7fc640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fb0078e80 0x7f8fb007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.242+0000 7f8fe5948640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fb0078e80 0x7f8fb007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.242+0000 7f8fe5948640 10 monclient: get_auth_request con 0x7f8fb0078e80 auth_method 0 2026-02-22T23:23:31.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.243+0000 7f8fce7fc640 10 client.4566.objecter ms_handle_connect 0x7f8fb0078e80 2026-02-22T23:23:31.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.243+0000 7f8fe5948640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fb0078e80 0x7f8fb007b340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009770 tx=0x7f8fd0006d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.244+0000 7f8fce7fc640 1 -- 10.20.193.6:0/2517309369 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f8fdc05de10 con 0x7f8fe006da30 2026-02-22T23:23:31.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.244+0000 7f8fce7fc640 10 client.4566.objecter ms_dispatch2 0x7f8fe0000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.244+0000 7f8fce7fc640 3 client.4566.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.244+0000 7f8fce7fc640 3 client.4566.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.244+0000 7f8fce7fc640 20 client.4566.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.246+0000 7f1d77fff640 1 -- 10.20.193.6:0/363057409 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1d88007030 con 0x7f1d80003ba0 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f0c3ae8f640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f0c3ae8f640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f0c3ae8f640 10 monclient: monmap: 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.247796+0000 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.247796+0000 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.248 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f1d77fff640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d6c078e00 0x7f1d6c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f1d9affd640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d6c078e00 0x7f1d6c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f1d9affd640 10 monclient: get_auth_request con 0x7f1d6c078e00 auth_method 0 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f0c3ae8f640 1 Processor -- start 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f0c3ae8f640 1 -- start start 2026-02-22T23:23:31.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f0c3ae8f640 10 monclient: init 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f1d77fff640 10 client.4570.objecter ms_handle_connect 0x7f1d6c078e00 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.247+0000 7f1d9affd640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d6c078e00 0x7f1d6c07b2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0c7a70 tx=0x7f1d8c002e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient: _add_conns ranks=[3,1,4,0,5,2] 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f0c34155f80 0x7f0c3415c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient(hunting): picked mon.noname-a con 0x7f0c34155f80 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0c340ccb40 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f0c3415da30 0x7f0c340caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient(hunting): picked mon.noname-b con 0x7f0c3415da30 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340d0380 con 0x7f0c340ccb40 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3406d5b0 con 0x7f0c34155f80 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3406c9a0 con 0x7f0c3415da30 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.248+0000 7f0c3ae8f640 10 monclient(hunting): authenticate will time out at 1085.975252s 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.250+0000 7f1d77fff640 1 -- 10.20.193.6:0/363057409 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f1d880824e0 con 0x7f1d80003ba0 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.250+0000 7f1d77fff640 10 client.4570.objecter ms_dispatch2 0x7f1d9c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.250+0000 7f1d77fff640 3 client.4570.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.250+0000 7f1d77fff640 3 client.4570.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.250+0000 7f1d77fff640 20 client.4570.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.257+0000 7f1d77fff640 1 -- 10.20.193.6:0/363057409 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1d88053080 con 0x7f1d80003ba0 2026-02-22T23:23:31.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.257+0000 7f1d77fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.257+0000 7f1d77fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.261+0000 7fa19bfff640 1 -- 10.20.193.6:0/3117020877 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1b0066640 con 0x7fa1c014ff60 2026-02-22T23:23:31.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.261+0000 7fa19bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 7}] 2026-02-22T23:23:31.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.261+0000 7fa19bfff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.263 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:31.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.263+0000 7f0c39405640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f0c3415da30 0x7f0c340caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.263+0000 7f0c2bfff640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f0c340ccb40 0x7f0c3415d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:60424/0 (socket says 10.20.193.6:60424) 2026-02-22T23:23:31.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.263+0000 7f0c2bfff640 1 -- 10.20.193.6:0/339738629 learned_addr learned my addr 10.20.193.6:0/339738629 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 10 librados: watch_flush enter 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 10 librados: watch_flush exit 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 20 client.4564.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 10 client.4564.objecter successfully canceled tick 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa190078e30 msgr2=0x7fa19007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa190078e30 0x7fa19007b2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1c015af60 tx=0x7fa1ac033000 comp rx=0 tx=0).stop 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 10 monclient: shutdown 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c014ff60 msgr2=0x7fa1c014e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c014ff60 0x7fa1c014e200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000e890 tx=0x7fa1b000ed60 comp rx=0 tx=0).stop 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 shutdown_connections 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa190078e30 0x7fa19007b2f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1c015d5e0 0x7fa1c014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1c014ff60 0x7fa1c014e200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 --2- 10.20.193.6:0/3117020877 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1c006ff20 0x7fa1c006ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 >> 10.20.193.6:0/3117020877 conn(0x7fa1c00cd780 msgr2=0x7fa1c00c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 shutdown_connections 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 -- 10.20.193.6:0/3117020877 wait complete. 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.264+0000 7fa1c6061640 1 librados: shutdown 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.265+0000 7f0c39405640 10 monclient(hunting): get_auth_request con 0x7f0c3415da30 auth_method 0 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.265+0000 7f0c39405640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.265+0000 7f0c39405640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.265+0000 7f0c39405640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.266+0000 7f0c39405640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.266+0000 7f0c39405640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.266+0000 7f0c39405640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f5b90e2b640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f5b90e2b640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f5b90e2b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 13} v 0) -- 0x7f5b8c0c6fc0 con 0x7f5b8c06ff20 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 10 monclient(hunting): handle_auth_done global_id 4573 payload 274 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 1 -- 10.20.193.6:0/339738629 >> v2:10.20.193.6:3300/0 conn(0x7f0c34155f80 msgr2=0x7f0c3415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 1 --2- 10.20.193.6:0/339738629 >> v2:10.20.193.6:3300/0 conn(0x7f0c34155f80 0x7f0c3415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 1 -- 10.20.193.6:0/339738629 >> v1:10.20.193.40:6789/0 conn(0x7f0c340ccb40 legacy=0x7f0c3415d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.267+0000 7f0c39405640 1 -- 10.20.193.6:0/339738629 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3406cc80 con 0x7f0c3415da30 2026-02-22T23:23:31.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.270+0000 7f0c39405640 1 --2- 10.20.193.6:0/339738629 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c3415da30 0x7f0c340caf30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0c340c6570 tx=0x7f0c3002f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83168173e8c1d7e4 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.7 2026-02-22T23:23:31.275 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 1 -- 10.20.193.6:0/339738629 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3003c070 con 0x7f0c3415da30 2026-02-22T23:23:31.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: dump: 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 1 -- 10.20.193.6:0/339738629 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3002fe40 con 0x7f0c3415da30 2026-02-22T23:23:31.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 1 -- 10.20.193.6:0/339738629 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30037660 con 0x7f0c3415da30 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c3ae8f640 5 monclient: authenticate success, global_id 4573 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c3ae8f640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c3ae8f640 4 set_mon_vals no callback set 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.273+0000 7f0c2b7fe640 10 monclient: dump: 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 10 monclient: shutdown 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/339738629 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c3415da30 msgr2=0x7f0c340caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/339738629 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c3415da30 0x7f0c340caf30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0c340c6570 tx=0x7f0c3002f130 comp rx=0 tx=0).stop 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/339738629 shutdown_connections 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/339738629 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c3415da30 0x7f0c340caf30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/339738629 >> v2:10.20.193.6:3300/0 conn(0x7f0c34155f80 0x7f0c3415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/339738629 >> 10.20.193.6:0/339738629 conn(0x7f0c340cc020 msgr2=0x7f0c340cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/339738629 shutdown_connections 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/339738629 wait complete. 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 10 monclient: monmap: 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.280 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 librados: starting msgr at 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 librados: starting objecter 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 Processor -- start 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 -- start start 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 librados: setting wanted keys 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 1 librados: calling monclient init 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.274+0000 7f0c3ae8f640 10 monclient: init 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c340ccb40 0x7f0c340cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): picked mon.noname-a con 0x7f0c340ccb40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c34155f80 0x7f0c341f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): picked mon.noname-b con 0x7f0c34155f80 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c341f7640 0x7f0c34071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): picked mon.noname-c con 0x7f0c341f7640 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34072eb0 con 0x7f0c340ccb40 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3406d240 con 0x7f0c34155f80 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34156360 con 0x7f0c341f7640 2026-02-22T23:23:31.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c3ae8f640 10 monclient(hunting): authenticate will time out at 1086.002390s 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c2bfff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c34155f80 0x7f0c341f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c2bfff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c34155f80 0x7f0c341f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36894/0 (socket says 10.20.193.6:36894) 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c2bfff640 1 -- 10.20.193.6:0/4162696487 learned_addr learned my addr 10.20.193.6:0/4162696487 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c2bfff640 10 monclient(hunting): get_auth_request con 0x7f0c34155f80 auth_method 0 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c2bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c2bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.275+0000 7f0c2bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.276+0000 7f0c2bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.276+0000 7f0c2bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.276+0000 7f0c2bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 10 monclient(hunting): handle_auth_done global_id 4576 payload 824 2026-02-22T23:23:31.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 1 -- 10.20.193.6:0/4162696487 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c341f7640 msgr2=0x7f0c34071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c341f7640 0x7f0c34071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 1 -- 10.20.193.6:0/4162696487 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c340ccb40 msgr2=0x7f0c340cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c340ccb40 0x7f0c340cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.277+0000 7f0c2bfff640 1 -- 10.20.193.6:0/4162696487 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c340ce270 con 0x7f0c34155f80 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.278+0000 7f0c2bfff640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c34155f80 0x7f0c341f7100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c730 tx=0x7f0c1c00cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.278+0000 7f0c297fa640 10 client.?.objecter ms_handle_connect 0x7f0c34155f80 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.278+0000 7f0c297fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 1 -- 10.20.193.6:0/4162696487 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c017070 con 0x7f0c34155f80 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: dump: 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.284 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 1 -- 10.20.193.6:0/4162696487 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1c005b90 con 0x7f0c34155f80 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 1 -- 10.20.193.6:0/4162696487 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c007070 con 0x7f0c34155f80 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c297fa640 10 monclient: dump: 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 5 monclient: authenticate success, global_id 4576 2026-02-22T23:23:31.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c2affd640 4 set_mon_vals no callback set 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 10 monclient: _renew_subs 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c340ce550 con 0x7f0c34155f80 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 10 client.4576.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 10 monclient: _renew_subs 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c340ceb10 con 0x7f0c34155f80 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.280+0000 7f0c3ae8f640 1 librados: init done 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.281+0000 7f0c3ae8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.281+0000 7f0c3ae8f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.281+0000 7f0c3ae8f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.281+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c3415c880 con 0x7f0c34155f80 2026-02-22T23:23:31.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.292+0000 7f0c297fa640 1 -- 10.20.193.6:0/4162696487 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0c1c007210 con 0x7f0c34155f80 2026-02-22T23:23:31.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.292+0000 7f0c297fa640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c04078e00 0x7f0c0407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.292+0000 7f9d62f44640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:31.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.292+0000 7f9d62f44640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:31.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.292+0000 7f9d62f44640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.292+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 12} v 0) -- 0x7f9d5c1562f0 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.293+0000 7f0c38c04640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c04078e00 0x7f0c0407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.293+0000 7f0c38c04640 10 monclient: get_auth_request con 0x7f0c04078e00 auth_method 0 2026-02-22T23:23:31.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.293+0000 7f0c38c04640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c04078e00 0x7f0c0407b2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c240098a0 tx=0x7f0c24006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.294+0000 7f0c297fa640 10 client.4576.objecter ms_handle_connect 0x7f0c04078e00 2026-02-22T23:23:31.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.294+0000 7f0c297fa640 1 -- 10.20.193.6:0/4162696487 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f0c1c05d510 con 0x7f0c34155f80 2026-02-22T23:23:31.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.294+0000 7f0c297fa640 10 client.4576.objecter ms_dispatch2 0x7f0c34000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.294+0000 7f0c297fa640 3 client.4576.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.294+0000 7f0c297fa640 3 client.4576.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.294+0000 7f0c297fa640 20 client.4576.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.306+0000 7fa1f6ffd640 1 -- 10.20.193.6:0/2487973943 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa20400f460 con 0x7fa210072e50 2026-02-22T23:23:31.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.306+0000 7fa1f6ffd640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1dc078e00 0x7fa1dc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.307+0000 7fa20dd74640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1dc078e00 0x7fa1dc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.307+0000 7fa20dd74640 10 monclient: get_auth_request con 0x7fa1dc078e00 auth_method 0 2026-02-22T23:23:31.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.307+0000 7fa20dd74640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1dc078e00 0x7fa1dc07b2c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa210073cd0 tx=0x7fa1f8006cf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.307+0000 7fa1f6ffd640 10 client.4634.objecter ms_handle_connect 0x7fa1dc078e00 2026-02-22T23:23:31.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.309+0000 7fa1f6ffd640 1 -- 10.20.193.6:0/2487973943 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fa20400c250 con 0x7fa210072e50 2026-02-22T23:23:31.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.309+0000 7fa1f6ffd640 10 client.4634.objecter ms_dispatch2 0x7fa210000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.309+0000 7fa1f6ffd640 3 client.4634.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.309+0000 7fa1f6ffd640 3 client.4634.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.309+0000 7fa1f6ffd640 20 client.4634.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.316+0000 7fa1f6ffd640 1 -- 10.20.193.6:0/2487973943 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa20401d020 con 0x7fa210072e50 2026-02-22T23:23:31.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.316+0000 7fa1f6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.316+0000 7fa1f6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.322+0000 7f5b72ffd640 1 -- 10.20.193.6:0/3196546808 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 13}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f5b800620e0 con 0x7f5b8c06ff20 2026-02-22T23:23:31.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.322+0000 7f5b72ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 13}] 2026-02-22T23:23:31.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.322+0000 7f5b72ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.323 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:31.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 10 librados: watch_flush enter 2026-02-22T23:23:31.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 10 librados: watch_flush exit 2026-02-22T23:23:31.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 20 client.4625.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 10 client.4625.objecter successfully canceled tick 2026-02-22T23:23:31.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b58078e50 msgr2=0x7f5b5807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b58078e50 0x7f5b5807b310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b74039000 tx=0x7f5b74031000 comp rx=0 tx=0).stop 2026-02-22T23:23:31.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 10 monclient: shutdown 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c06ff20 msgr2=0x7f5b8c1f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c06ff20 0x7f5b8c1f51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c730 tx=0x7f5b8000cc00 comp rx=0 tx=0).stop 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 shutdown_connections 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b58078e50 0x7f5b5807b310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5b8c15a5e0 0x7f5b8c06e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5b8c0c99a0 0x7f5b8c06e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 --2- 10.20.193.6:0/3196546808 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5b8c06ff20 0x7f5b8c1f51c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 >> 10.20.193.6:0/3196546808 conn(0x7f5b8c0cd780 msgr2=0x7f5b8c14e620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 shutdown_connections 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 -- 10.20.193.6:0/3196546808 wait complete. 2026-02-22T23:23:31.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.324+0000 7f5b90e2b640 1 librados: shutdown 2026-02-22T23:23:31.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.13 2026-02-22T23:23:31.336 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.343+0000 7f1da20a9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:31.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.343+0000 7f1da20a9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:31.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.343+0000 7f1da20a9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.343+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1d9c15a3c0 con 0x7f1d80003ba0 2026-02-22T23:23:31.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.344+0000 7f8fce7fc640 1 -- 10.20.193.6:0/2517309369 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8fdc01a060 con 0x7f8fe006da30 2026-02-22T23:23:31.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.344+0000 7f8fce7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.344+0000 7f8fce7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.351+0000 7f9d517fa640 1 -- 10.20.193.6:0/2473396999 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 12}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f9d440871e0 con 0x7f9d5c1f7bb0 2026-02-22T23:23:31.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.351+0000 7f9d517fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 12}] 2026-02-22T23:23:31.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.351+0000 7f9d517fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.353 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:31.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 10 librados: watch_flush enter 2026-02-22T23:23:31.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 10 librados: watch_flush exit 2026-02-22T23:23:31.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 20 client.4563.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 10 client.4563.objecter successfully canceled tick 2026-02-22T23:23:31.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9d2c078e30 msgr2=0x7f9d2c07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9d2c078e30 0x7f9d2c07b2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0097d0 tx=0x7f9d4c006db0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 10 monclient: shutdown 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d5c1f7bb0 msgr2=0x7f9d5c157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d5c1f7bb0 0x7f9d5c157ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d44002410 tx=0x7f9d440048d0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 shutdown_connections 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9d2c078e30 0x7f9d2c07b2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9d5c158000 0x7f9d5c0bc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9d5c1f7bb0 0x7f9d5c157ac0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 --2- 10.20.193.6:0/2473396999 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9d48004900 0x7f9d5c147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 >> 10.20.193.6:0/2473396999 conn(0x7f9d5c0cd780 msgr2=0x7f9d5c14fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 shutdown_connections 2026-02-22T23:23:31.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 -- 10.20.193.6:0/2473396999 wait complete. 2026-02-22T23:23:31.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.354+0000 7f9d62f44640 1 librados: shutdown 2026-02-22T23:23:31.358 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:23:31.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.12 2026-02-22T23:23:31.365 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.397+0000 7f0c297fa640 1 -- 10.20.193.6:0/4162696487 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0c1c021020 con 0x7f0c34155f80 2026-02-22T23:23:31.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.397+0000 7f0c297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.397+0000 7f0c297fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.401+0000 7f1d77fff640 1 -- 10.20.193.6:0/363057409 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d9c148f10 con 0x7f1d80003ba0 2026-02-22T23:23:31.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.401+0000 7f1d77fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2026-02-22T23:23:31.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.401+0000 7f1d77fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.403 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:31.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.403+0000 7fa20f577640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:31.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.403+0000 7fa20f577640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:31.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.403+0000 7fa20f577640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.403+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa21006dd60 con 0x7fa210072e50 2026-02-22T23:23:31.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 10 librados: watch_flush enter 2026-02-22T23:23:31.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 10 librados: watch_flush exit 2026-02-22T23:23:31.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 20 client.4570.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 10 client.4570.objecter successfully canceled tick 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d6c078e00 msgr2=0x7f1d6c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d6c078e00 0x7f1d6c07b2c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0c7a70 tx=0x7f1d8c002e70 comp rx=0 tx=0).stop 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 10 monclient: shutdown 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 msgr2=0x7f1d9c148550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d9c148550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802ece0 tx=0x7f1d880027c0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 shutdown_connections 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d6c078e00 0x7f1d6c07b2c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d9c073b40 0x7f1d9c1ecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d9c148a90 0x7f1d9c073600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 --2- 10.20.193.6:0/363057409 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d80003ba0 0x7f1d9c148550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 >> 10.20.193.6:0/363057409 conn(0x7f1d9c0cd780 msgr2=0x7f1d9c158010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 shutdown_connections 2026-02-22T23:23:31.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 -- 10.20.193.6:0/363057409 wait complete. 2026-02-22T23:23:31.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.404+0000 7f1da20a9640 1 librados: shutdown 2026-02-22T23:23:31.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.406+0000 7fa1f6ffd640 1 -- 10.20.193.6:0/2487973943 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fa204062710 con 0x7fa210072e50 2026-02-22T23:23:31.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.406+0000 7fa1f6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 11}] 2026-02-22T23:23:31.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.406+0000 7fa1f6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.409 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:31.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 10 librados: watch_flush enter 2026-02-22T23:23:31.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 10 librados: watch_flush exit 2026-02-22T23:23:31.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 20 client.4634.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 10 client.4634.objecter successfully canceled tick 2026-02-22T23:23:31.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1dc078e00 msgr2=0x7fa1dc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1dc078e00 0x7fa1dc07b2c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa210073cd0 tx=0x7fa1f8006cf0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 10 monclient: shutdown 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa210072e50 msgr2=0x7fa2101f7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa210072e50 0x7fa2101f7780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa20400b570 tx=0x7fa20400ba40 comp rx=0 tx=0).stop 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 shutdown_connections 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1dc078e00 0x7fa1dc07b2c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa210155a50 0x7fa2101f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa2100c9160 0x7fa2101f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 --2- 10.20.193.6:0/2487973943 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa210072e50 0x7fa2101f7780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 >> 10.20.193.6:0/2487973943 conn(0x7fa2100cd480 msgr2=0x7fa2100cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 shutdown_connections 2026-02-22T23:23:31.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.408+0000 7fa20f577640 1 -- 10.20.193.6:0/2487973943 wait complete. 2026-02-22T23:23:31.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.409+0000 7fa20f577640 1 librados: shutdown 2026-02-22T23:23:31.416 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.3 2026-02-22T23:23:31.416 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.11 2026-02-22T23:23:31.419 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.428+0000 7ff49a7e3640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.428+0000 7ff49a7e3640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.428+0000 7ff49a7e3640 10 monclient: monmap: 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.428980+0000 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.428980+0000 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.430 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.431 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.431 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.431 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.428+0000 7ff49a7e3640 1 Processor -- start 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.428+0000 7ff49a7e3640 1 -- start start 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.428+0000 7ff49a7e3640 10 monclient: init 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient: _add_conns ranks=[5,1,0,2,4,3] 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff49406ff20 0x7ff494070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient(hunting): picked mon.noname-c con 0x7ff49406ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff49415c970 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:31.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff494155eb0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4940725b0 con 0x7ff494155eb0 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff494072890 con 0x7ff49415c970 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff494072b70 con 0x7ff49406ff20 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff49a7e3640 10 monclient(hunting): authenticate will time out at 1086.156524s 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff498d59640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7ff494155eb0 0x7ff4940c7d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35594/0 (socket says 10.20.193.6:35594) 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff498d59640 1 -- 10.20.193.6:0/1330242388 learned_addr learned my addr 10.20.193.6:0/1330242388 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.429+0000 7ff493fff640 1 --2- 10.20.193.6:0/1330242388 >> v2:10.20.193.80:3300/0 conn(0x7ff49406ff20 0x7ff494070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7ff493fff640 10 monclient(hunting): get_auth_request con 0x7ff49406ff20 auth_method 0 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7ff493fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7ff493fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7ff493fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7f8fe7bd3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:31.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7f8fe7bd3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:31.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7f8fe7bd3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8fe015c750 con 0x7f8fe006da30 2026-02-22T23:23:31.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7ff493fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7ff493fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.430+0000 7ff493fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 10 monclient(hunting): handle_auth_done global_id 4637 payload 274 2026-02-22T23:23:31.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff492ffd640 1 -- 10.20.193.6:0/1330242388 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650202219 0 0) 0x7ff4940725b0 con 0x7ff494155eb0 2026-02-22T23:23:31.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 1 -- 10.20.193.6:0/1330242388 >> v1:10.20.193.40:6789/0 conn(0x7ff49415c970 legacy=0x7ff4941557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 1 -- 10.20.193.6:0/1330242388 >> v1:10.20.193.6:6789/0 conn(0x7ff494155eb0 legacy=0x7ff4940c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 1 monclient: found mon.noname-c 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff493fff640 1 -- 10.20.193.6:0/1330242388 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff494072e50 con 0x7ff49406ff20 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.431+0000 7ff492ffd640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.432+0000 7ff493fff640 1 --2- 10.20.193.6:0/1330242388 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49406ff20 0x7ff494070300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff48002ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e83b0f0959e94bf server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 1 -- 10.20.193.6:0/1330242388 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48003c070 con 0x7ff49406ff20 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: dump: 2026-02-22T23:23:31.439 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 1 -- 10.20.193.6:0/1330242388 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48002fad0 con 0x7ff49406ff20 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff49a7e3640 5 monclient: authenticate success, global_id 4637 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff49a7e3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 1 -- 10.20.193.6:0/1330242388 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480037440 con 0x7ff49406ff20 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff492ffd640 10 monclient: dump: 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.441 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff49a7e3640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.433+0000 7ff49a7e3640 4 set_mon_vals no callback set 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 10 monclient: shutdown 2026-02-22T23:23:31.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 1 -- 10.20.193.6:0/1330242388 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49406ff20 msgr2=0x7ff494070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/1330242388 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49406ff20 0x7ff494070300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4800098a0 tx=0x7ff48002ee10 comp rx=0 tx=0).stop 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 1 -- 10.20.193.6:0/1330242388 shutdown_connections 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/1330242388 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49406ff20 0x7ff494070300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 1 -- 10.20.193.6:0/1330242388 >> 10.20.193.6:0/1330242388 conn(0x7ff4940cd780 msgr2=0x7ff4940cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 1 -- 10.20.193.6:0/1330242388 shutdown_connections 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 1 -- 10.20.193.6:0/1330242388 wait complete. 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.434+0000 7ff49a7e3640 10 monclient: monmap: 2026-02-22T23:23:31.443 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.444 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7f8fce7fc640 1 -- 10.20.193.6:0/2517309369 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7f8fdc01ce20 con 0x7f8fe006da30 2026-02-22T23:23:31.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7f8fce7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 10}] 2026-02-22T23:23:31.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7f8fce7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 librados: starting msgr at 2026-02-22T23:23:31.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 librados: starting objecter 2026-02-22T23:23:31.446 INFO:teuthology.orchestra.run.trial006.stdout:90194313219 2026-02-22T23:23:31.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 Processor -- start 2026-02-22T23:23:31.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 -- start start 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 librados: setting wanted keys 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 librados: calling monclient init 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient: init 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff49406ff20 0x7ff49414f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): picked mon.noname-b con 0x7ff49406ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff494155eb0 0x7ff4941f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): picked mon.noname-a con 0x7ff494155eb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49415c970 0x7ff4941e7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): picked mon.noname-c con 0x7ff49415c970 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415d340 con 0x7ff494155eb0 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415d1c0 con 0x7ff49406ff20 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff49415d4c0 con 0x7ff49415c970 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff49a7e3640 10 monclient(hunting): authenticate will time out at 1086.162695s 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.435+0000 7ff4937fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff494155eb0 0x7ff4941f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49415c970 0x7ff4941e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff494155eb0 0x7ff4941f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40174/0 (socket says 10.20.193.6:40174) 2026-02-22T23:23:31.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 1 -- 10.20.193.6:0/3588487606 learned_addr learned my addr 10.20.193.6:0/3588487606 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 10 monclient(hunting): get_auth_request con 0x7ff494155eb0 auth_method 0 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 10 monclient(hunting): get_auth_request con 0x7ff49415c970 auth_method 0 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff498d59640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7ff4937fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7f8fe7bd3640 10 librados: watch_flush enter 2026-02-22T23:23:31.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.436+0000 7f8fe7bd3640 10 librados: watch_flush exit 2026-02-22T23:23:31.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 20 client.4566.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 10 client.4566.objecter successfully canceled tick 2026-02-22T23:23:31.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fb0078e80 msgr2=0x7f8fb007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fb0078e80 0x7f8fb007b340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009770 tx=0x7f8fd0006d40 comp rx=0 tx=0).stop 2026-02-22T23:23:31.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 10 monclient: shutdown 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fe006da30 msgr2=0x7f8fe006dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fe006da30 0x7f8fe006dee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00c4410 tx=0x7f8fdc00a430 comp rx=0 tx=0).stop 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 shutdown_connections 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fb0078e80 0x7f8fb007b340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8fe006da30 0x7f8fe006dee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8fe01e9970 0x7f8fe0153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 --2- 10.20.193.6:0/2517309369 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8fc8004900 0x7f8fe0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 >> 10.20.193.6:0/2517309369 conn(0x7f8fe00cd780 msgr2=0x7f8fe00c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 shutdown_connections 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 -- 10.20.193.6:0/2517309369 wait complete. 2026-02-22T23:23:31.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7f8fe7bd3640 1 librados: shutdown 2026-02-22T23:23:31.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 10 monclient(hunting): handle_auth_done global_id 4640 payload 824 2026-02-22T23:23:31.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 1 -- 10.20.193.6:0/3588487606 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff49406ff20 msgr2=0x7ff49414f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff49406ff20 0x7ff49414f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 1 -- 10.20.193.6:0/3588487606 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff494155eb0 msgr2=0x7ff4941f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff494155eb0 0x7ff4941f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 1 monclient: found mon.noname-c 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.437+0000 7ff498d59640 1 -- 10.20.193.6:0/3588487606 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4941e7b70 con 0x7ff49415c970 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.438+0000 7ff498d59640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49415c970 0x7ff4941e7270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff48800ca50 tx=0x7ff48800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.438+0000 7ff490ff9640 10 client.?.objecter ms_handle_connect 0x7ff49415c970 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.438+0000 7ff490ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff490ff9640 1 -- 10.20.193.6:0/3588487606 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488018070 con 0x7ff49415c970 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff490ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff490ff9640 10 monclient: dump: 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.457 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff490ff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff490ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff490ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 5 monclient: authenticate success, global_id 4640 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 10 monclient: _renew_subs 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4941e7e50 con 0x7ff49415c970 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 10 client.4640.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 10 monclient: _renew_subs 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4941e8410 con 0x7ff49415c970 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.439+0000 7ff49a7e3640 1 librados: init done 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff490ff9640 1 -- 10.20.193.6:0/3588487606 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48800abe0 con 0x7ff49415c970 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff490ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff490ff9640 1 -- 10.20.193.6:0/3588487606 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488022060 con 0x7ff49415c970 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff490ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff4927fc640 4 set_mon_vals no callback set 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff490ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff490ff9640 10 monclient: dump: 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.459 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff49a7e3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff49a7e3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff49a7e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.440+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff494070300 con 0x7ff49415c970 2026-02-22T23:23:31.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.10 2026-02-22T23:23:31.461 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.450+0000 7ff490ff9640 1 -- 10.20.193.6:0/3588487606 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff488025080 con 0x7ff49415c970 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.451+0000 7ff490ff9640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff460078e00 0x7ff46007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.451+0000 7ff493fff640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff460078e00 0x7ff46007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.451+0000 7ff493fff640 10 monclient: get_auth_request con 0x7ff460078e00 auth_method 0 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.452+0000 7ff493fff640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff460078e00 0x7ff46007b2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff480039000 tx=0x7ff480031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.452+0000 7ff490ff9640 10 client.4640.objecter ms_handle_connect 0x7ff460078e00 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.452+0000 7ff490ff9640 1 -- 10.20.193.6:0/3588487606 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7ff48805df30 con 0x7ff49415c970 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.452+0000 7ff490ff9640 10 client.4640.objecter ms_dispatch2 0x7ff494000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.452+0000 7ff490ff9640 3 client.4640.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.452+0000 7ff490ff9640 3 client.4640.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.452+0000 7ff490ff9640 20 client.4640.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.468 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-22T23:23:31.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.484+0000 7f0c3ae8f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:31.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.484+0000 7f0c3ae8f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:31.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.484+0000 7f0c3ae8f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.484+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0c340c7d60 con 0x7f0c34155f80 2026-02-22T23:23:31.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.486+0000 7f0c297fa640 1 -- 10.20.193.6:0/4162696487 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0c1c061bd0 con 0x7f0c34155f80 2026-02-22T23:23:31.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.486+0000 7f0c297fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 6}] 2026-02-22T23:23:31.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.486+0000 7f0c297fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.488 INFO:teuthology.orchestra.run.trial006.stdout:90194313220 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 10 librados: watch_flush enter 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 10 librados: watch_flush exit 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 20 client.4576.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 10 client.4576.objecter successfully canceled tick 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c04078e00 msgr2=0x7f0c0407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c04078e00 0x7f0c0407b2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0c240098a0 tx=0x7f0c24006d90 comp rx=0 tx=0).stop 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 10 monclient: shutdown 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c34155f80 msgr2=0x7f0c341f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c34155f80 0x7f0c341f7100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c730 tx=0x7f0c1c00cc00 comp rx=0 tx=0).stop 2026-02-22T23:23:31.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 shutdown_connections 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c04078e00 0x7f0c0407b2c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c341f7640 0x7f0c34071160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c34155f80 0x7f0c341f7100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 --2- 10.20.193.6:0/4162696487 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c340ccb40 0x7f0c340cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 >> 10.20.193.6:0/4162696487 conn(0x7f0c340cc020 msgr2=0x7f0c340c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 shutdown_connections 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 -- 10.20.193.6:0/4162696487 wait complete. 2026-02-22T23:23:31.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.488+0000 7f0c3ae8f640 1 librados: shutdown 2026-02-22T23:23:31.497 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2026-02-22T23:23:31.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.6 2026-02-22T23:23:31.498 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.547+0000 7ff490ff9640 1 -- 10.20.193.6:0/3588487606 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff488061ef0 con 0x7ff49415c970 2026-02-22T23:23:31.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.547+0000 7ff490ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.547+0000 7ff490ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.551+0000 7f88ed445640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.551+0000 7f88ed445640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.551+0000 7f88ed445640 10 monclient: monmap: 2026-02-22T23:23:31.552 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.552275+0000 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.552275+0000 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.552+0000 7f88ed445640 1 Processor -- start 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.552+0000 7f88ed445640 1 -- start start 2026-02-22T23:23:31.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.552+0000 7f88ed445640 10 monclient: init 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient: _add_conns ranks=[3,5,0,2,4,1] 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f88e81527f0 0x7f88e8073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient(hunting): picked mon.noname-a con 0x7f88e81527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f88e815d2c0 0x7f88e815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient(hunting): picked mon.noname-c con 0x7f88e815d2c0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f88e815d730 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e8073c20 con 0x7f88e815d730 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f88e8074b10 con 0x7f88e81527f0 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f88e8073f00 con 0x7f88e815d2c0 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88ed445640 10 monclient(hunting): authenticate will time out at 1086.280119s 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88e6ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f88e81527f0 0x7f88e8073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88e6ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f88e81527f0 0x7f88e8073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40176/0 (socket says 10.20.193.6:40176) 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88e6ffd640 1 -- 10.20.193.6:0/2541950451 learned_addr learned my addr 10.20.193.6:0/2541950451 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88e6ffd640 10 monclient(hunting): get_auth_request con 0x7f88e81527f0 auth_method 0 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88e6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.553+0000 7f88e6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 1 --2- 10.20.193.6:0/2541950451 >> v2:10.20.193.80:3300/0 conn(0x7f88e815d2c0 0x7f88e815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 10 monclient(hunting): get_auth_request con 0x7f88e815d2c0 auth_method 0 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.554+0000 7f88e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_done global_id 4575 payload 274 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 1 -- 10.20.193.6:0/2541950451 >> v2:10.20.193.80:3300/0 conn(0x7f88e815d2c0 msgr2=0x7f88e815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 1 --2- 10.20.193.6:0/2541950451 >> v2:10.20.193.80:3300/0 conn(0x7f88e815d2c0 0x7f88e815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 1 -- 10.20.193.6:0/2541950451 >> v1:10.20.193.6:6789/0 conn(0x7f88e815d730 legacy=0x7f88e80ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 1 monclient: found mon.noname-a 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.555+0000 7f88e6ffd640 1 -- 10.20.193.6:0/2541950451 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e80741e0 con 0x7f88e81527f0 2026-02-22T23:23:31.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.557+0000 7f88e6ffd640 1 --2- 10.20.193.6:0/2541950451 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e8073530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88e808f510 tx=0x7f88dc02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3122a7166f8585b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 1 -- 10.20.193.6:0/2541950451 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc03c070 con 0x7f88e81527f0 2026-02-22T23:23:31.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: dump: 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88ed445640 5 monclient: authenticate success, global_id 4575 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 1 -- 10.20.193.6:0/2541950451 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc02fbe0 con 0x7f88e81527f0 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88ed445640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 1 -- 10.20.193.6:0/2541950451 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc037440 con 0x7f88e81527f0 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88ed445640 4 set_mon_vals no callback set 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.559+0000 7f88e5ffb640 10 monclient: dump: 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.561 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.562 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.562 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.562 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 10 monclient: shutdown 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 -- 10.20.193.6:0/2541950451 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 msgr2=0x7f88e8073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 --2- 10.20.193.6:0/2541950451 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e8073530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f88e808f510 tx=0x7f88dc02eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 -- 10.20.193.6:0/2541950451 shutdown_connections 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 --2- 10.20.193.6:0/2541950451 >> v2:10.20.193.80:3300/0 conn(0x7f88e815d2c0 0x7f88e815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 --2- 10.20.193.6:0/2541950451 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e8073530 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 -- 10.20.193.6:0/2541950451 >> 10.20.193.6:0/2541950451 conn(0x7f88e80ce730 msgr2=0x7f88e80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 -- 10.20.193.6:0/2541950451 shutdown_connections 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 -- 10.20.193.6:0/2541950451 wait complete. 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 10 monclient: monmap: 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.563 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 librados: starting msgr at 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.560+0000 7f88ed445640 1 librados: starting objecter 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 Processor -- start 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 -- start start 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 librados: setting wanted keys 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 librados: calling monclient init 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient: init 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): picked mon.noname-a con 0x7f88e81527f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f88e815d2c0 0x7f88e81f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): picked mon.noname-c con 0x7f88e815d2c0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88e815d730 0x7f88e8148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): picked mon.noname-b con 0x7f88e815d730 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8074fa0 con 0x7f88e81527f0 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f88e80747a0 con 0x7f88e815d730 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8075120 con 0x7f88e815d2c0 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88ed445640 10 monclient(hunting): authenticate will time out at 1086.288612s 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88e6ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.561+0000 7f88e67fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f88e815d2c0 0x7f88e81f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e814eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40186/0 (socket says 10.20.193.6:40186) 2026-02-22T23:23:31.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 1 -- 10.20.193.6:0/3626062566 learned_addr learned my addr 10.20.193.6:0/3626062566 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 10 monclient(hunting): get_auth_request con 0x7f88e81527f0 auth_method 0 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88e815d730 0x7f88e8148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e67fc640 10 monclient(hunting): get_auth_request con 0x7f88e815d2c0 auth_method 0 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 10 monclient(hunting): get_auth_request con 0x7f88e815d730 auth_method 0 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.562+0000 7f88e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 10 monclient(hunting): handle_auth_done global_id 4581 payload 824 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 1 -- 10.20.193.6:0/3626062566 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f88e815d2c0 msgr2=0x7f88e81f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f88e815d2c0 0x7f88e81f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 1 -- 10.20.193.6:0/3626062566 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88e815d730 msgr2=0x7f88e8148390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88e815d730 0x7f88e8148390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 1 monclient: found mon.noname-a 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.563+0000 7f88e6ffd640 1 -- 10.20.193.6:0/3626062566 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e81f4370 con 0x7f88e81527f0 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.564+0000 7f88e6ffd640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e814eb40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f88dc02f480 tx=0x7f88dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.564+0000 7f88c77fe640 10 client.?.objecter ms_handle_connect 0x7f88e81527f0 2026-02-22T23:23:31.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.564+0000 7f88c77fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 1 -- 10.20.193.6:0/3626062566 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc03c040 con 0x7f88e81527f0 2026-02-22T23:23:31.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:31.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: dump: 2026-02-22T23:23:31.567 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 1 -- 10.20.193.6:0/3626062566 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc042630 con 0x7f88e81527f0 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 1 -- 10.20.193.6:0/3626062566 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc042a20 con 0x7f88e81527f0 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 5 monclient: authenticate success, global_id 4581 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 10 monclient: _renew_subs 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88e57fa640 4 set_mon_vals no callback set 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e8158e30 con 0x7f88e81527f0 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88c77fe640 10 monclient: dump: 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.568 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 10 client.4581.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 10 monclient: _renew_subs 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81593f0 con 0x7f88e81527f0 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.566+0000 7f88ed445640 1 librados: init done 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.567+0000 7f88ed445640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.567+0000 7f88ed445640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.567+0000 7f88ed445640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.567+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e8159630 con 0x7f88e81527f0 2026-02-22T23:23:31.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.605+0000 7fef83acd640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.605+0000 7fef83acd640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.605+0000 7fef83acd640 10 monclient: monmap: 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.606009+0000 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.606009+0000 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.607 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.608 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.608 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.608 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.608 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.608 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.605+0000 7fef83acd640 1 Processor -- start 2026-02-22T23:23:31.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.605+0000 7fef83acd640 1 -- start start 2026-02-22T23:23:31.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.605+0000 7fef83acd640 10 monclient: init 2026-02-22T23:23:31.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient: _add_conns ranks=[4,0,1,5,2,3] 2026-02-22T23:23:31.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fef7c155fb0 0x7fef7c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient(hunting): picked mon.noname-b con 0x7fef7c155fb0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fef7c15dfd0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fef7c15f310 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c06f050 con 0x7fef7c15dfd0 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c06f330 con 0x7fef7c15f310 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fef7c15cce0 con 0x7fef7c155fb0 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef83acd640 10 monclient(hunting): authenticate will time out at 1086.333506s 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.606+0000 7fef81842640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fef7c155fb0 0x7fef7c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fef7c155fb0 0x7fef7c15c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36922/0 (socket says 10.20.193.6:36922) 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 1 -- 10.20.193.6:0/4177167608 learned_addr learned my addr 10.20.193.6:0/4177167608 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 10 monclient(hunting): get_auth_request con 0x7fef7c155fb0 auth_method 0 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.607+0000 7fef81842640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 10 monclient(hunting): handle_auth_done global_id 4582 payload 274 2026-02-22T23:23:31.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 1 -- 10.20.193.6:0/4177167608 >> v1:10.20.193.40:6789/0 conn(0x7fef7c15f310 legacy=0x7fef7c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 1 -- 10.20.193.6:0/4177167608 >> v1:10.20.193.6:6789/0 conn(0x7fef7c15dfd0 legacy=0x7fef7c15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:31.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.608+0000 7fef81842640 1 -- 10.20.193.6:0/4177167608 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c15d210 con 0x7fef7c155fb0 2026-02-22T23:23:31.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.609+0000 7fef81842640 1 --2- 10.20.193.6:0/4177167608 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c155fb0 0x7fef7c15c7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0099a0 tx=0x7fef6c02f030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808f32a2704fd83d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef80840640 1 -- 10.20.193.6:0/4177167608 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c03c070 con 0x7fef7c155fb0 2026-02-22T23:23:31.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef80840640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef80840640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef80840640 10 monclient: dump: 2026-02-22T23:23:31.617 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.617 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef80840640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef80840640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef80840640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef83acd640 5 monclient: authenticate success, global_id 4582 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.610+0000 7fef83acd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef80840640 1 -- 10.20.193.6:0/4177167608 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6c02fc60 con 0x7fef7c155fb0 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef80840640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef80840640 1 -- 10.20.193.6:0/4177167608 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c037660 con 0x7fef7c155fb0 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef80840640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef80840640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef80840640 10 monclient: dump: 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.619 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef83acd640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.611+0000 7fef83acd640 4 set_mon_vals no callback set 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 10 monclient: shutdown 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 -- 10.20.193.6:0/4177167608 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c155fb0 msgr2=0x7fef7c15c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 --2- 10.20.193.6:0/4177167608 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c155fb0 0x7fef7c15c7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0099a0 tx=0x7fef6c02f030 comp rx=0 tx=0).stop 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 -- 10.20.193.6:0/4177167608 shutdown_connections 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 --2- 10.20.193.6:0/4177167608 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c155fb0 0x7fef7c15c7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 -- 10.20.193.6:0/4177167608 >> 10.20.193.6:0/4177167608 conn(0x7fef7c06c820 msgr2=0x7fef7c06cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 -- 10.20.193.6:0/4177167608 shutdown_connections 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 -- 10.20.193.6:0/4177167608 wait complete. 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 10 monclient: monmap: 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.621 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 librados: starting msgr at 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 librados: starting objecter 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 Processor -- start 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 -- start start 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 librados: setting wanted keys 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 1 librados: calling monclient init 2026-02-22T23:23:31.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.612+0000 7fef83acd640 10 monclient: init 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef7c155fb0 0x7fef7c1f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): picked mon.noname-c con 0x7fef7c155fb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef7c15dfd0 0x7fef7c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): picked mon.noname-a con 0x7fef7c15dfd0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c15f310 0x7fef7c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): picked mon.noname-b con 0x7fef7c15f310 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c15dab0 con 0x7fef7c15dfd0 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c15d7d0 con 0x7fef7c15f310 2026-02-22T23:23:31.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c156390 con 0x7fef7c155fb0 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef83acd640 10 monclient(hunting): authenticate will time out at 1086.340294s 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c15f310 0x7fef7c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef81842640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef7c155fb0 0x7fef7c1f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c15f310 0x7fef7c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36930/0 (socket says 10.20.193.6:36930) 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 1 -- 10.20.193.6:0/2230548945 learned_addr learned my addr 10.20.193.6:0/2230548945 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 10 monclient(hunting): get_auth_request con 0x7fef7c15f310 auth_method 0 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef81842640 10 monclient(hunting): get_auth_request con 0x7fef7c155fb0 auth_method 0 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef81842640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef81842640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef81842640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.613+0000 7fef82043640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.614+0000 7fef82043640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.614+0000 7fef81842640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.614+0000 7fef81842640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.614+0000 7fef81842640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 10 monclient(hunting): handle_auth_done global_id 4585 payload 824 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 1 -- 10.20.193.6:0/2230548945 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef7c155fb0 msgr2=0x7fef7c1f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef7c155fb0 0x7fef7c1f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 1 -- 10.20.193.6:0/2230548945 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef7c15dfd0 msgr2=0x7fef7c14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef7c15dfd0 0x7fef7c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.615+0000 7fef82043640 1 -- 10.20.193.6:0/2230548945 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c1f8270 con 0x7fef7c15f310 2026-02-22T23:23:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.616+0000 7fef82043640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c15f310 0x7fef7c0ca700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef7800cc40 tx=0x7fef7800a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.616+0000 7fef6a7fc640 10 client.?.objecter ms_handle_connect 0x7fef7c15f310 2026-02-22T23:23:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.616+0000 7fef6a7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef6a7fc640 1 -- 10.20.193.6:0/2230548945 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78016020 con 0x7fef7c15f310 2026-02-22T23:23:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef6a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef6a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef6a7fc640 10 monclient: dump: 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef6a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef6a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef6a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 5 monclient: authenticate success, global_id 4585 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 monclient: _renew_subs 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c0cb170 con 0x7fef7c15f310 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 client.4585.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 monclient: _renew_subs 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c0cb730 con 0x7fef7c15f310 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 1 librados: init done 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.617+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c15c7a0 con 0x7fef7c15f310 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.618+0000 7fef6a7fc640 1 -- 10.20.193.6:0/2230548945 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef78025440 con 0x7fef7c15f310 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.618+0000 7fef6a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.618+0000 7fef6a7fc640 1 -- 10.20.193.6:0/2230548945 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7800ce60 con 0x7fef7c15f310 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.618+0000 7fef6a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.618+0000 7fef6a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.618+0000 7fef6bfff640 4 set_mon_vals no callback set 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.618+0000 7fef6a7fc640 10 monclient: dump: 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.631 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.632 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.632 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.632 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.632 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.633+0000 7ff49a7e3640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:31.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.633+0000 7ff49a7e3640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:31.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.633+0000 7ff49a7e3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:31.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.633+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff49415c750 con 0x7ff49415c970 2026-02-22T23:23:31.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.635+0000 7ff490ff9640 1 -- 10.20.193.6:0/3588487606 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4880661f0 con 0x7ff49415c970 2026-02-22T23:23:31.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.635+0000 7ff490ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2026-02-22T23:23:31.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.635+0000 7ff490ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.637 INFO:teuthology.orchestra.run.trial006.stdout:98784247812 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 10 librados: watch_flush enter 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 10 librados: watch_flush exit 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 20 client.4640.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 10 client.4640.objecter successfully canceled tick 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff460078e00 msgr2=0x7ff46007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff460078e00 0x7ff46007b2c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff480039000 tx=0x7ff480031000 comp rx=0 tx=0).stop 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 10 monclient: shutdown 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49415c970 msgr2=0x7ff4941e7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49415c970 0x7ff4941e7270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff48800ca50 tx=0x7ff48800cf20 comp rx=0 tx=0).stop 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 shutdown_connections 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff460078e00 0x7ff46007b2c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff49415c970 0x7ff4941e7270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff494155eb0 0x7ff4941f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 --2- 10.20.193.6:0/3588487606 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff49406ff20 0x7ff49414f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 >> 10.20.193.6:0/3588487606 conn(0x7ff4940cd780 msgr2=0x7ff494150b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 shutdown_connections 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 -- 10.20.193.6:0/3588487606 wait complete. 2026-02-22T23:23:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.637+0000 7ff49a7e3640 1 librados: shutdown 2026-02-22T23:23:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.639+0000 7f88c77fe640 1 -- 10.20.193.6:0/3626062566 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f88dc042bc0 con 0x7f88e81527f0 2026-02-22T23:23:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.639+0000 7f88c77fe640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f88b4078e50 0x7f88b407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.640+0000 7f88e67fc640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f88b4078e50 0x7f88b407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.640+0000 7f88e67fc640 10 monclient: get_auth_request con 0x7f88b4078e50 auth_method 0 2026-02-22T23:23:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.641+0000 7f88e67fc640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f88b4078e50 0x7f88b407b310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f88e814fbb0 tx=0x7f88d0009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.641+0000 7f88c77fe640 10 client.4581.objecter ms_handle_connect 0x7f88b4078e50 2026-02-22T23:23:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.643+0000 7f88c77fe640 1 -- 10.20.193.6:0/3626062566 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f88dc02faa0 con 0x7f88e81527f0 2026-02-22T23:23:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.643+0000 7f88c77fe640 10 client.4581.objecter ms_dispatch2 0x7f88e8000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.643+0000 7f88c77fe640 3 client.4581.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.643+0000 7f88c77fe640 3 client.4581.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.643+0000 7f88c77fe640 20 client.4581.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.0 2026-02-22T23:23:31.647 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.682+0000 7f88c77fe640 1 -- 10.20.193.6:0/3626062566 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f88e8159630 con 0x7f88e81527f0 2026-02-22T23:23:31.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.682+0000 7f88c77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.682+0000 7f88c77fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.723+0000 7fef6a7fc640 1 -- 10.20.193.6:0/2230548945 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fef78012420 con 0x7fef7c15f310 2026-02-22T23:23:31.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.724+0000 7fef6a7fc640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef4c078e00 0x7fef4c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.724+0000 7fef81842640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef4c078e00 0x7fef4c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.724+0000 7fef81842640 10 monclient: get_auth_request con 0x7fef4c078e00 auth_method 0 2026-02-22T23:23:31.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.724+0000 7fef6a7fc640 10 client.4585.objecter ms_handle_connect 0x7fef4c078e00 2026-02-22T23:23:31.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.724+0000 7fef81842640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef4c078e00 0x7fef4c07b2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009970 tx=0x7fef6c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.725+0000 7fef6a7fc640 1 -- 10.20.193.6:0/2230548945 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7fef7805e8a0 con 0x7fef7c15f310 2026-02-22T23:23:31.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.725+0000 7fef6a7fc640 10 client.4585.objecter ms_dispatch2 0x7fef7c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:31.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.725+0000 7fef6a7fc640 3 client.4585.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:31.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.725+0000 7fef6a7fc640 3 client.4585.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:31.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.726+0000 7fef6a7fc640 20 client.4585.objecter dump_active .. 0 homeless 2026-02-22T23:23:31.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.732+0000 7fef6a7fc640 1 -- 10.20.193.6:0/2230548945 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fef78020020 con 0x7fef7c15f310 2026-02-22T23:23:31.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.732+0000 7fef6a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.732+0000 7fef6a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:31.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.766+0000 7f88ed445640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:31.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.766+0000 7f88ed445640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:31.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.766+0000 7f88ed445640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:31.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.766+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f88e815dd20 con 0x7f88e81527f0 2026-02-22T23:23:31.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.769+0000 7f88c77fe640 1 -- 10.20.193.6:0/3626062566 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88e81f4370 con 0x7f88e81527f0 2026-02-22T23:23:31.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.769+0000 7f88c77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2026-02-22T23:23:31.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.769+0000 7f88c77fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.770 INFO:teuthology.orchestra.run.trial006.stdout:94489280516 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 10 librados: watch_flush enter 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 10 librados: watch_flush exit 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 20 client.4581.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 10 client.4581.objecter successfully canceled tick 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f88b4078e50 msgr2=0x7f88b407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f88b4078e50 0x7f88b407b310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f88e814fbb0 tx=0x7f88d0009290 comp rx=0 tx=0).stop 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 10 monclient: shutdown 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 msgr2=0x7f88e814eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e814eb40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f88dc02f480 tx=0x7f88dc0047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 shutdown_connections 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f88b4078e50 0x7f88b407b310 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f88e815d730 0x7f88e8148390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f88e815d2c0 0x7f88e81f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 --2- 10.20.193.6:0/3626062566 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f88e81527f0 0x7f88e814eb40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 >> 10.20.193.6:0/3626062566 conn(0x7f88e80ce730 msgr2=0x7f88e815e520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 shutdown_connections 2026-02-22T23:23:31.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 -- 10.20.193.6:0/3626062566 wait complete. 2026-02-22T23:23:31.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.771+0000 7f88ed445640 1 librados: shutdown 2026-02-22T23:23:31.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.5 2026-02-22T23:23:31.781 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.816+0000 7fef83acd640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:31.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.816+0000 7fef83acd640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:31.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.816+0000 7fef83acd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.816+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 15} v 0) -- 0x7fef7c0cbe70 con 0x7fef7c15f310 2026-02-22T23:23:31.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.819+0000 7fef6a7fc640 1 -- 10.20.193.6:0/2230548945 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 15}]=0 v0) ==== 75+0+12 (secure 0 0 0) 0x7fef7c1f8270 con 0x7fef7c15f310 2026-02-22T23:23:31.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.819+0000 7fef6a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 15}] 2026-02-22T23:23:31.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.819+0000 7fef6a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:31.820 INFO:teuthology.orchestra.run.trial006.stdout:94489280517 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 10 librados: watch_flush enter 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 10 librados: watch_flush exit 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 20 client.4585.objecter shutdown clearing up homeless session... 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 10 client.4585.objecter successfully canceled tick 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef4c078e00 msgr2=0x7fef4c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef4c078e00 0x7fef4c07b2c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009970 tx=0x7fef6c033000 comp rx=0 tx=0).stop 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 10 monclient: shutdown 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c15f310 msgr2=0x7fef7c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c15f310 0x7fef7c0ca700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef7800cc40 tx=0x7fef7800a430 comp rx=0 tx=0).stop 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 shutdown_connections 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef4c078e00 0x7fef4c07b2c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef7c15f310 0x7fef7c0ca700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef7c15dfd0 0x7fef7c14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 --2- 10.20.193.6:0/2230548945 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef7c155fb0 0x7fef7c1f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 >> 10.20.193.6:0/2230548945 conn(0x7fef7c06c820 msgr2=0x7fef7c0c7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 shutdown_connections 2026-02-22T23:23:31.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 -- 10.20.193.6:0/2230548945 wait complete. 2026-02-22T23:23:31.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.820+0000 7fef83acd640 1 librados: shutdown 2026-02-22T23:23:31.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.15 2026-02-22T23:23:31.830 DEBUG:teuthology.parallel:result is None 2026-02-22T23:23:31.830 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:23:31.830 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:23:31.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.933+0000 7f65a1511640 10 monclient: get_monmap_and_config 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.933+0000 7f65a1511640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.933+0000 7f65a1511640 10 monclient: monmap: 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:31.935333+0000 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:31.935333+0000 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 1 Processor -- start 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 1 -- start start 2026-02-22T23:23:31.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient: init 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient: _add_conns ranks=[0,4,2,1,3,5] 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f659c06ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f659c0c99a0 0x7f659c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient(hunting): picked mon.noname-b con 0x7f659c0c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f659c15a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f659c0c7e90 con 0x7f659c06ff20 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f659c0c8170 con 0x7f659c15a5e0 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f659c159a40 con 0x7f659c0c99a0 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.934+0000 7f65a1511640 10 monclient(hunting): authenticate will time out at 1086.662714s 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659affd640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f659c06ff20 0x7f659c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35614/0 (socket says 10.20.193.6:35614) 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659affd640 1 -- 10.20.193.6:0/1828018733 learned_addr learned my addr 10.20.193.6:0/1828018733 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 1 --2- 10.20.193.6:0/1828018733 >> v2:10.20.193.40:3300/0 conn(0x7f659c0c99a0 0x7f659c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 10 monclient(hunting): get_auth_request con 0x7f659c0c99a0 auth_method 0 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.935+0000 7f659a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f6599ffb640 1 -- 10.20.193.6:0/1828018733 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1088154437 0 0) 0x7f659c0c7e90 con 0x7f659c06ff20 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f6599ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f6599ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f6599ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f6599ffb640 10 monclient(hunting): my global_id is 4584 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f6599ffb640 1 -- 10.20.193.6:0/1828018733 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6588003620 con 0x7f659c06ff20 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 10 monclient(hunting): handle_auth_done global_id 4588 payload 274 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 1 -- 10.20.193.6:0/1828018733 >> v1:10.20.193.80:6789/0 conn(0x7f659c15a5e0 legacy=0x7f659c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 1 -- 10.20.193.6:0/1828018733 >> v1:10.20.193.6:6789/0 conn(0x7f659c06ff20 legacy=0x7f659c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.936+0000 7f659a7fc640 1 -- 10.20.193.6:0/1828018733 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c0c8450 con 0x7f659c0c99a0 2026-02-22T23:23:31.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.937+0000 7f659a7fc640 1 --2- 10.20.193.6:0/1828018733 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c0c99a0 0x7f659c070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f65840098d0 tx=0x7f658402eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5075d95410482769 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 1 -- 10.20.193.6:0/1828018733 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658403c070 con 0x7f659c0c99a0 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: dump: 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 1 -- 10.20.193.6:0/1828018733 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f658402fba0 con 0x7f659c0c99a0 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 5 monclient: authenticate success, global_id 4588 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:31.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 1 -- 10.20.193.6:0/1828018733 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584037440 con 0x7f659c0c99a0 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 4 set_mon_vals no callback set 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f6599ffb640 10 monclient: dump: 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 10 monclient: shutdown 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 -- 10.20.193.6:0/1828018733 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c0c99a0 msgr2=0x7f659c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 --2- 10.20.193.6:0/1828018733 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c0c99a0 0x7f659c070300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f65840098d0 tx=0x7f658402eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 -- 10.20.193.6:0/1828018733 shutdown_connections 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 --2- 10.20.193.6:0/1828018733 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c0c99a0 0x7f659c070300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 -- 10.20.193.6:0/1828018733 >> 10.20.193.6:0/1828018733 conn(0x7f659c0cd780 msgr2=0x7f659c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 -- 10.20.193.6:0/1828018733 shutdown_connections 2026-02-22T23:23:31.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 -- 10.20.193.6:0/1828018733 wait complete. 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 10 monclient: build_initial_monmap 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 10 monclient: monmap: 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 librados: starting msgr at 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.939+0000 7f65a1511640 1 librados: starting objecter 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 Processor -- start 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 -- start start 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 librados: setting wanted keys 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 librados: calling monclient init 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient: init 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:31.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c06ff20 0x7f659c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): picked mon.noname-b con 0x7f659c06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f659c0c99a0 0x7f659c1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): picked mon.noname-a con 0x7f659c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f659c15a5e0 0x7f659c154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): picked mon.noname-c con 0x7f659c15a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f659c150520 con 0x7f659c0c99a0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0c8a10 con 0x7f659c06ff20 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1506a0 con 0x7f659c15a5e0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f65a1511640 10 monclient(hunting): authenticate will time out at 1086.668464s 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659a7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f659c0c99a0 0x7f659c1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c06ff20 0x7f659c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659b7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f659c15a5e0 0x7f659c154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659affd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c06ff20 0x7f659c157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36950/0 (socket says 10.20.193.6:36950) 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659affd640 1 -- 10.20.193.6:0/2466485715 learned_addr learned my addr 10.20.193.6:0/2466485715 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659affd640 10 monclient(hunting): get_auth_request con 0x7f659c06ff20 auth_method 0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.940+0000 7f659affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659b7fe640 10 monclient(hunting): get_auth_request con 0x7f659c15a5e0 auth_method 0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659a7fc640 10 monclient(hunting): get_auth_request con 0x7f659c0c99a0 auth_method 0 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:31.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.941+0000 7f659a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 10 monclient(hunting): handle_auth_done global_id 4591 payload 824 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 1 -- 10.20.193.6:0/2466485715 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f659c15a5e0 msgr2=0x7f659c154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f659c15a5e0 0x7f659c154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 1 -- 10.20.193.6:0/2466485715 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f659c0c99a0 msgr2=0x7f659c1f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f659c0c99a0 0x7f659c1f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 1 monclient: found mon.noname-b 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.942+0000 7f659affd640 1 -- 10.20.193.6:0/2466485715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c154f40 con 0x7f659c06ff20 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.943+0000 7f659affd640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c06ff20 0x7f659c157190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f659000c730 tx=0x7f659000cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.943+0000 7f657b7fe640 10 client.?.objecter ms_handle_connect 0x7f659c06ff20 2026-02-22T23:23:31.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.943+0000 7f657b7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f657b7fe640 1 -- 10.20.193.6:0/2466485715 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590017070 con 0x7f659c06ff20 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f657b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f657b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f657b7fe640 10 monclient: dump: 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f657b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f657b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:31.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f657b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 5 monclient: authenticate success, global_id 4591 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 monclient: _renew_subs 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c155220 con 0x7f659c06ff20 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 client.4591.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 monclient: _renew_subs 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1557e0 con 0x7f659c06ff20 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 1 librados: init done 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.944+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c070300 con 0x7f659c06ff20 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.945+0000 7f657b7fe640 1 -- 10.20.193.6:0/2466485715 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590005bd0 con 0x7f659c06ff20 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.945+0000 7f657b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.945+0000 7f657b7fe640 1 -- 10.20.193.6:0/2466485715 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590007070 con 0x7f659c06ff20 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.945+0000 7f657b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.945+0000 7f657b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.945+0000 7f65997fa640 4 set_mon_vals no callback set 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:31.945+0000 7f657b7fe640 10 monclient: dump: 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:31.947 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.050+0000 7f657b7fe640 1 -- 10.20.193.6:0/2466485715 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6590007210 con 0x7f659c06ff20 2026-02-22T23:23:32.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.051+0000 7f657b7fe640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6568078e00 0x7f656807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.051+0000 7f659a7fc640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6568078e00 0x7f656807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.051+0000 7f659a7fc640 10 monclient: get_auth_request con 0x7f6568078e00 auth_method 0 2026-02-22T23:23:32.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.051+0000 7f659a7fc640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6568078e00 0x7f656807b2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6584039000 tx=0x7f6584033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:32.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.051+0000 7f657b7fe640 10 client.4591.objecter ms_handle_connect 0x7f6568078e00 2026-02-22T23:23:32.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.052+0000 7f657b7fe640 1 -- 10.20.193.6:0/2466485715 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f659005d4c0 con 0x7f659c06ff20 2026-02-22T23:23:32.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.052+0000 7f657b7fe640 10 client.4591.objecter ms_dispatch2 0x7f659c000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:32.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.052+0000 7f657b7fe640 3 client.4591.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:32.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.052+0000 7f657b7fe640 3 client.4591.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:32.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.052+0000 7f657b7fe640 20 client.4591.objecter dump_active .. 0 homeless 2026-02-22T23:23:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.059+0000 7f657b7fe640 1 -- 10.20.193.6:0/2466485715 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6590022020 con 0x7f659c06ff20 2026-02-22T23:23:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.059+0000 7f657b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.059+0000 7f657b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:32.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.139+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f659c0c6d50 con 0x7f6568078e00 2026-02-22T23:23:32.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.140+0000 7f657b7fe640 1 -- 10.20.193.6:0/2466485715 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45511 (secure 0 0 0) 0x7f659c0c6d50 con 0x7f6568078e00 2026-02-22T23:23:32.144 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 10 librados: watch_flush enter 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 10 librados: watch_flush exit 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 20 client.4591.objecter shutdown clearing up homeless session... 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 10 client.4591.objecter successfully canceled tick 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6568078e00 msgr2=0x7f656807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6568078e00 0x7f656807b2c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6584039000 tx=0x7f6584033000 comp rx=0 tx=0).stop 2026-02-22T23:23:32.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 10 monclient: shutdown 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c06ff20 msgr2=0x7f659c157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c06ff20 0x7f659c157190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f659000c730 tx=0x7f659000cc00 comp rx=0 tx=0).stop 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 shutdown_connections 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6568078e00 0x7f656807b2c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f659c15a5e0 0x7f659c154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f659c0c99a0 0x7f659c1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 --2- 10.20.193.6:0/2466485715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f659c06ff20 0x7f659c157190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 >> 10.20.193.6:0/2466485715 conn(0x7f659c0cd780 msgr2=0x7f659c14e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 shutdown_connections 2026-02-22T23:23:32.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 -- 10.20.193.6:0/2466485715 wait complete. 2026-02-22T23:23:32.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.141+0000 7f65a1511640 1 librados: shutdown 2026-02-22T23:23:32.151 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2026-02-22T23:23:31.756598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328211,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":43,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":436692,"kb_used_data":5200,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512748,"statfs":{"total":1546188226560,"available":1545741053952,"internally_reserved":0,"allocated":5324800,"data_stored":2493462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25502,"internal_metadata":441424994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000737"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.541669+0000","last_change":"2026-02-22T23:23:26.541669+0000","last_active":"2026-02-22T23:23:26.541669+0000","last_peered":"2026-02-22T23:23:26.541669+0000","last_clean":"2026-02-22T23:23:26.541669+0000","last_became_active":"2026-02-22T23:23:24.999148+0000","last_became_peered":"2026-02-22T23:23:24.999148+0000","last_unstale":"2026-02-22T23:23:26.541669+0000","last_undegraded":"2026-02-22T23:23:26.541669+0000","last_fullsized":"2026-02-22T23:23:26.541669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:33:54.198723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033743088999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.928396+0000","last_change":"2026-02-22T23:23:26.948831+0000","last_active":"2026-02-22T23:23:26.928396+0000","last_peered":"2026-02-22T23:23:26.928396+0000","last_clean":"2026-02-22T23:23:26.928396+0000","last_became_active":"2026-02-22T23:23:24.966693+0000","last_became_peered":"2026-02-22T23:23:24.966693+0000","last_unstale":"2026-02-22T23:23:26.928396+0000","last_undegraded":"2026-02-22T23:23:26.928396+0000","last_fullsized":"2026-02-22T23:23:26.928396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:30:16.311433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033152011000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.516263+0000","last_change":"2026-02-22T23:23:26.535885+0000","last_active":"2026-02-22T23:23:26.516263+0000","last_peered":"2026-02-22T23:23:26.516263+0000","last_clean":"2026-02-22T23:23:26.516263+0000","last_became_active":"2026-02-22T23:23:24.806260+0000","last_became_peered":"2026-02-22T23:23:24.806260+0000","last_unstale":"2026-02-22T23:23:26.516263+0000","last_undegraded":"2026-02-22T23:23:26.516263+0000","last_fullsized":"2026-02-22T23:23:26.516263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:46:46.747409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032290068999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.524413+0000","last_change":"2026-02-22T23:23:26.547663+0000","last_active":"2026-02-22T23:23:26.524413+0000","last_peered":"2026-02-22T23:23:26.524413+0000","last_clean":"2026-02-22T23:23:26.524413+0000","last_became_active":"2026-02-22T23:23:24.706162+0000","last_became_peered":"2026-02-22T23:23:24.706162+0000","last_unstale":"2026-02-22T23:23:26.524413+0000","last_undegraded":"2026-02-22T23:23:26.524413+0000","last_fullsized":"2026-02-22T23:23:26.524413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:37:21.100237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041859805999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"27'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.397225+0000","last_change":"2026-02-22T23:23:26.386245+0000","last_active":"2026-02-22T23:23:26.397225+0000","last_peered":"2026-02-22T23:23:26.397225+0000","last_clean":"2026-02-22T23:23:26.397225+0000","last_became_active":"2026-02-22T23:23:25.054022+0000","last_became_peered":"2026-02-22T23:23:25.054022+0000","last_unstale":"2026-02-22T23:23:26.397225+0000","last_undegraded":"2026-02-22T23:23:26.397225+0000","last_fullsized":"2026-02-22T23:23:26.397225+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:02:19.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.070943361999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.542103+0000","last_change":"2026-02-22T23:23:26.542103+0000","last_active":"2026-02-22T23:23:26.542103+0000","last_peered":"2026-02-22T23:23:26.542103+0000","last_clean":"2026-02-22T23:23:26.542103+0000","last_became_active":"2026-02-22T23:23:24.462347+0000","last_became_peered":"2026-02-22T23:23:24.462347+0000","last_unstale":"2026-02-22T23:23:26.542103+0000","last_undegraded":"2026-02-22T23:23:26.542103+0000","last_fullsized":"2026-02-22T23:23:26.542103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:09:34.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033702211000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.382023+0000","last_change":"2026-02-22T23:23:26.382023+0000","last_active":"2026-02-22T23:23:26.382023+0000","last_peered":"2026-02-22T23:23:26.382023+0000","last_clean":"2026-02-22T23:23:26.382023+0000","last_became_active":"2026-02-22T23:23:24.464964+0000","last_became_peered":"2026-02-22T23:23:24.464964+0000","last_unstale":"2026-02-22T23:23:26.382023+0000","last_undegraded":"2026-02-22T23:23:26.382023+0000","last_fullsized":"2026-02-22T23:23:26.382023+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:28:59.922786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055238717,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"25'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.538271+0000","last_change":"2026-02-22T23:23:26.538271+0000","last_active":"2026-02-22T23:23:26.538271+0000","last_peered":"2026-02-22T23:23:26.538271+0000","last_clean":"2026-02-22T23:23:26.538271+0000","last_became_active":"2026-02-22T23:23:24.503509+0000","last_became_peered":"2026-02-22T23:23:24.503509+0000","last_unstale":"2026-02-22T23:23:26.538271+0000","last_undegraded":"2026-02-22T23:23:26.538271+0000","last_fullsized":"2026-02-22T23:23:26.538271+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:47:22.484108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032869134000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'19","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.500949+0000","last_change":"2026-02-22T23:23:22.760169+0000","last_active":"2026-02-22T23:23:26.500949+0000","last_peered":"2026-02-22T23:23:26.500949+0000","last_clean":"2026-02-22T23:23:26.500949+0000","last_became_active":"2026-02-22T23:23:22.740754+0000","last_became_peered":"2026-02-22T23:23:22.740754+0000","last_unstale":"2026-02-22T23:23:26.500949+0000","last_undegraded":"2026-02-22T23:23:26.500949+0000","last_fullsized":"2026-02-22T23:23:26.500949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_clean_scrub_stamp":"2026-02-22T23:23:20.759065+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:14:46.099842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":41,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":41,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":656384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,9,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2,4,5,6,7,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,3,4,5,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":6,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,4,5,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[2,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":294912,"data_stored":114834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[0,4,6,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,5,6,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":294912,"data_stored":114834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[2,4,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,5,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":622592,"data_stored":443007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":622592,"data_stored":443007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":328192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":328192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:23:32.154 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:23:32.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.221+0000 7f93858ea640 10 monclient: get_monmap_and_config 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.221+0000 7f93858ea640 10 monclient: build_initial_monmap 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.221+0000 7f93858ea640 10 monclient: monmap: 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:32.223163+0000 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:32.223163+0000 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.224 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:32.225 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:32.225 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:32.225 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:32.225 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:32.225 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:32.225 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 1 Processor -- start 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 1 -- start start 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient: init 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient: _add_conns ranks=[4,2,5,1,3,0] 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f938006ff20 0x7f9380070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient(hunting): picked mon.noname-b con 0x7f938006ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f93800c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:32.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f938015a5e0 0x7f938014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient(hunting): picked mon.noname-c con 0x7f938015a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93800c7e90 con 0x7f93800c99a0 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9380159a40 con 0x7f938006ff20 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f93800c8170 con 0x7f938015a5e0 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.222+0000 7f93858ea640 10 monclient(hunting): authenticate will time out at 1086.950653s 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f938006ff20 0x7f9380070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937f7fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f938015a5e0 0x7f938014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937e7fc640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f93800c99a0 0x7f9380159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36308/0 (socket says 10.20.193.6:36308) 2026-02-22T23:23:32.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f938006ff20 0x7f9380070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36952/0 (socket says 10.20.193.6:36952) 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937e7fc640 1 -- 10.20.193.6:0/1214746140 learned_addr learned my addr 10.20.193.6:0/1214746140 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 10 monclient(hunting): get_auth_request con 0x7f938006ff20 auth_method 0 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937f7fe640 10 monclient(hunting): get_auth_request con 0x7f938015a5e0 auth_method 0 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.223+0000 7f937f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 10 monclient(hunting): handle_auth_done global_id 4594 payload 274 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 1 -- 10.20.193.6:0/1214746140 >> v2:10.20.193.80:3300/0 conn(0x7f938015a5e0 msgr2=0x7f938014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 1 --2- 10.20.193.6:0/1214746140 >> v2:10.20.193.80:3300/0 conn(0x7f938015a5e0 0x7f938014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 1 -- 10.20.193.6:0/1214746140 >> v1:10.20.193.80:6789/0 conn(0x7f93800c99a0 legacy=0x7f9380159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 1 monclient: found mon.noname-b 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.224+0000 7f937effd640 1 -- 10.20.193.6:0/1214746140 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93800c8450 con 0x7f938006ff20 2026-02-22T23:23:32.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.225+0000 7f937effd640 1 --2- 10.20.193.6:0/1214746140 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f938006ff20 0x7f9380070300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f93740098a0 tx=0x7f937402ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aca5717090fb7d8 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:32.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 1 -- 10.20.193.6:0/1214746140 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937403c070 con 0x7f938006ff20 2026-02-22T23:23:32.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:32.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: dump: 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:32.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 1 -- 10.20.193.6:0/1214746140 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f937402fae0 con 0x7f938006ff20 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 5 monclient: authenticate success, global_id 4594 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 1 -- 10.20.193.6:0/1214746140 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374037440 con 0x7f938006ff20 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 4 set_mon_vals no callback set 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f937dffb640 10 monclient: dump: 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.240 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 10 monclient: shutdown 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 -- 10.20.193.6:0/1214746140 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f938006ff20 msgr2=0x7f9380070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 --2- 10.20.193.6:0/1214746140 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f938006ff20 0x7f9380070300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f93740098a0 tx=0x7f937402ee10 comp rx=0 tx=0).stop 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 -- 10.20.193.6:0/1214746140 shutdown_connections 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 --2- 10.20.193.6:0/1214746140 >> v2:10.20.193.80:3300/0 conn(0x7f938015a5e0 0x7f938014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 --2- 10.20.193.6:0/1214746140 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f938006ff20 0x7f9380070300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 -- 10.20.193.6:0/1214746140 >> 10.20.193.6:0/1214746140 conn(0x7f93800cd780 msgr2=0x7f93800cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 -- 10.20.193.6:0/1214746140 shutdown_connections 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.227+0000 7f93858ea640 1 -- 10.20.193.6:0/1214746140 wait complete. 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient: build_initial_monmap 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient: monmap: 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:32.242 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 librados: starting msgr at 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 librados: starting objecter 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 Processor -- start 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 -- start start 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 librados: setting wanted keys 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 librados: calling monclient init 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient: init 2026-02-22T23:23:32.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f938006ff20 0x7f93801f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): picked mon.noname-a con 0x7f938006ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f93800c99a0 0x7f93801f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): picked mon.noname-b con 0x7f93800c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f938015a5e0 0x7f9380072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): picked mon.noname-c con 0x7f938015a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9380150350 con 0x7f938006ff20 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f93800c8a10 con 0x7f93800c99a0 2026-02-22T23:23:32.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f93801504d0 con 0x7f938015a5e0 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.228+0000 7f93858ea640 10 monclient(hunting): authenticate will time out at 1086.956957s 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f93800c99a0 0x7f93801f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f938015a5e0 0x7f9380072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f938006ff20 0x7f93801f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f93800c99a0 0x7f93801f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:36958/0 (socket says 10.20.193.6:36958) 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 1 -- 10.20.193.6:0/3287640464 learned_addr learned my addr 10.20.193.6:0/3287640464 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 10 monclient(hunting): get_auth_request con 0x7f93800c99a0 auth_method 0 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 10 monclient(hunting): get_auth_request con 0x7f938015a5e0 auth_method 0 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 10 monclient(hunting): get_auth_request con 0x7f938006ff20 auth_method 0 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.229+0000 7f937effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 10 monclient(hunting): handle_auth_done global_id 4664 payload 824 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 1 -- 10.20.193.6:0/3287640464 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f93800c99a0 msgr2=0x7f93801f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f93800c99a0 0x7f93801f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 1 -- 10.20.193.6:0/3287640464 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f938006ff20 msgr2=0x7f93801f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f938006ff20 0x7f93801f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 1 monclient: found mon.noname-c 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:32.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.231+0000 7f937f7fe640 1 -- 10.20.193.6:0/3287640464 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93800c10d0 con 0x7f938015a5e0 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.232+0000 7f937f7fe640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f938015a5e0 0x7f9380072ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f937000eba0 tx=0x7f937000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.232+0000 7f9357fff640 10 client.?.objecter ms_handle_connect 0x7f938015a5e0 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.232+0000 7f9357fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 1 -- 10.20.193.6:0/3287640464 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370018020 con 0x7f938015a5e0 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: dump: 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.248 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 1 -- 10.20.193.6:0/3287640464 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f937000cd30 con 0x7f938015a5e0 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 5 monclient: authenticate success, global_id 4664 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 10 monclient: _renew_subs 2026-02-22T23:23:32.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93801f3c70 con 0x7f938015a5e0 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 1 -- 10.20.193.6:0/3287640464 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700146c0 con 0x7f938015a5e0 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f9357fff640 10 monclient: dump: 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.250 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f937d7fa640 4 set_mon_vals no callback set 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 10 client.4664.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 10 monclient: _renew_subs 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801f4220 con 0x7f938015a5e0 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.233+0000 7f93858ea640 1 librados: init done 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.234+0000 7f93858ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.234+0000 7f93858ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.234+0000 7f93858ea640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.234+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9380150860 con 0x7f938015a5e0 2026-02-22T23:23:32.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.246+0000 7f9357fff640 1 -- 10.20.193.6:0/3287640464 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9370014860 con 0x7f938015a5e0 2026-02-22T23:23:32.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.246+0000 7f9357fff640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9350078e50 0x7f935007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.247+0000 7f937effd640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9350078e50 0x7f935007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.247+0000 7f937effd640 10 monclient: get_auth_request con 0x7f9350078e50 auth_method 0 2026-02-22T23:23:32.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.248+0000 7f937effd640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9350078e50 0x7f935007b310 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9374039000 tx=0x7f9374033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:32.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.248+0000 7f9357fff640 10 client.4664.objecter ms_handle_connect 0x7f9350078e50 2026-02-22T23:23:32.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.341+0000 7f9357fff640 1 -- 10.20.193.6:0/3287640464 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f937005edd0 con 0x7f938015a5e0 2026-02-22T23:23:32.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.341+0000 7f9357fff640 10 client.4664.objecter ms_dispatch2 0x7f9380000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:32.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.341+0000 7f9357fff640 3 client.4664.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:32.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.341+0000 7f9357fff640 3 client.4664.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:32.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.341+0000 7f9357fff640 20 client.4664.objecter dump_active .. 0 homeless 2026-02-22T23:23:32.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.347+0000 7f9357fff640 1 -- 10.20.193.6:0/3287640464 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9370022030 con 0x7f938015a5e0 2026-02-22T23:23:32.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.347+0000 7f9357fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:32.350 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.347+0000 7f9357fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:32.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.430+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f938014f510 con 0x7f9350078e50 2026-02-22T23:23:32.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.431+0000 7f9357fff640 1 -- 10.20.193.6:0/3287640464 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45511 (secure 0 0 0) 0x7f938014f510 con 0x7f9350078e50 2026-02-22T23:23:32.434 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:32.436 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-22T23:23:32.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.432+0000 7f93858ea640 10 librados: watch_flush enter 2026-02-22T23:23:32.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.432+0000 7f93858ea640 10 librados: watch_flush exit 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.432+0000 7f93858ea640 20 client.4664.objecter shutdown clearing up homeless session... 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.432+0000 7f93858ea640 10 client.4664.objecter successfully canceled tick 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9350078e50 msgr2=0x7f935007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9350078e50 0x7f935007b310 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9374039000 tx=0x7f9374033000 comp rx=0 tx=0).stop 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 10 monclient: shutdown 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f938015a5e0 msgr2=0x7f9380072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f938015a5e0 0x7f9380072ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f937000eba0 tx=0x7f937000c490 comp rx=0 tx=0).stop 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 shutdown_connections 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9350078e50 0x7f935007b310 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f938015a5e0 0x7f9380072ae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f93800c99a0 0x7f93801f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 --2- 10.20.193.6:0/3287640464 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f938006ff20 0x7f93801f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 >> 10.20.193.6:0/3287640464 conn(0x7f93800cd780 msgr2=0x7f938014e580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:32.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 shutdown_connections 2026-02-22T23:23:32.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 -- 10.20.193.6:0/3287640464 wait complete. 2026-02-22T23:23:32.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.433+0000 7f93858ea640 1 librados: shutdown 2026-02-22T23:23:32.444 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2026-02-22T23:23:31.756598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":328211,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":43,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":436692,"kb_used_data":5200,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512748,"statfs":{"total":1546188226560,"available":1545741053952,"internally_reserved":0,"allocated":5324800,"data_stored":2493462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25502,"internal_metadata":441424994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000737"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.541669+0000","last_change":"2026-02-22T23:23:26.541669+0000","last_active":"2026-02-22T23:23:26.541669+0000","last_peered":"2026-02-22T23:23:26.541669+0000","last_clean":"2026-02-22T23:23:26.541669+0000","last_became_active":"2026-02-22T23:23:24.999148+0000","last_became_peered":"2026-02-22T23:23:24.999148+0000","last_unstale":"2026-02-22T23:23:26.541669+0000","last_undegraded":"2026-02-22T23:23:26.541669+0000","last_fullsized":"2026-02-22T23:23:26.541669+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:33:54.198723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033743088999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.928396+0000","last_change":"2026-02-22T23:23:26.948831+0000","last_active":"2026-02-22T23:23:26.928396+0000","last_peered":"2026-02-22T23:23:26.928396+0000","last_clean":"2026-02-22T23:23:26.928396+0000","last_became_active":"2026-02-22T23:23:24.966693+0000","last_became_peered":"2026-02-22T23:23:24.966693+0000","last_unstale":"2026-02-22T23:23:26.928396+0000","last_undegraded":"2026-02-22T23:23:26.928396+0000","last_fullsized":"2026-02-22T23:23:26.928396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:30:16.311433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033152011000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.516263+0000","last_change":"2026-02-22T23:23:26.535885+0000","last_active":"2026-02-22T23:23:26.516263+0000","last_peered":"2026-02-22T23:23:26.516263+0000","last_clean":"2026-02-22T23:23:26.516263+0000","last_became_active":"2026-02-22T23:23:24.806260+0000","last_became_peered":"2026-02-22T23:23:24.806260+0000","last_unstale":"2026-02-22T23:23:26.516263+0000","last_undegraded":"2026-02-22T23:23:26.516263+0000","last_fullsized":"2026-02-22T23:23:26.516263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:46:46.747409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032290068999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.524413+0000","last_change":"2026-02-22T23:23:26.547663+0000","last_active":"2026-02-22T23:23:26.524413+0000","last_peered":"2026-02-22T23:23:26.524413+0000","last_clean":"2026-02-22T23:23:26.524413+0000","last_became_active":"2026-02-22T23:23:24.706162+0000","last_became_peered":"2026-02-22T23:23:24.706162+0000","last_unstale":"2026-02-22T23:23:26.524413+0000","last_undegraded":"2026-02-22T23:23:26.524413+0000","last_fullsized":"2026-02-22T23:23:26.524413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:37:21.100237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041859805999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"27'2","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.397225+0000","last_change":"2026-02-22T23:23:26.386245+0000","last_active":"2026-02-22T23:23:26.397225+0000","last_peered":"2026-02-22T23:23:26.397225+0000","last_clean":"2026-02-22T23:23:26.397225+0000","last_became_active":"2026-02-22T23:23:25.054022+0000","last_became_peered":"2026-02-22T23:23:25.054022+0000","last_unstale":"2026-02-22T23:23:26.397225+0000","last_undegraded":"2026-02-22T23:23:26.397225+0000","last_fullsized":"2026-02-22T23:23:26.397225+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:02:19.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.070943361999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.542103+0000","last_change":"2026-02-22T23:23:26.542103+0000","last_active":"2026-02-22T23:23:26.542103+0000","last_peered":"2026-02-22T23:23:26.542103+0000","last_clean":"2026-02-22T23:23:26.542103+0000","last_became_active":"2026-02-22T23:23:24.462347+0000","last_became_peered":"2026-02-22T23:23:24.462347+0000","last_unstale":"2026-02-22T23:23:26.542103+0000","last_undegraded":"2026-02-22T23:23:26.542103+0000","last_fullsized":"2026-02-22T23:23:26.542103+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:09:34.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033702211000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.382023+0000","last_change":"2026-02-22T23:23:26.382023+0000","last_active":"2026-02-22T23:23:26.382023+0000","last_peered":"2026-02-22T23:23:26.382023+0000","last_clean":"2026-02-22T23:23:26.382023+0000","last_became_active":"2026-02-22T23:23:24.464964+0000","last_became_peered":"2026-02-22T23:23:24.464964+0000","last_unstale":"2026-02-22T23:23:26.382023+0000","last_undegraded":"2026-02-22T23:23:26.382023+0000","last_fullsized":"2026-02-22T23:23:26.382023+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:28:59.922786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055238717,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"25'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.538271+0000","last_change":"2026-02-22T23:23:26.538271+0000","last_active":"2026-02-22T23:23:26.538271+0000","last_peered":"2026-02-22T23:23:26.538271+0000","last_clean":"2026-02-22T23:23:26.538271+0000","last_became_active":"2026-02-22T23:23:24.503509+0000","last_became_peered":"2026-02-22T23:23:24.503509+0000","last_unstale":"2026-02-22T23:23:26.538271+0000","last_undegraded":"2026-02-22T23:23:26.538271+0000","last_fullsized":"2026-02-22T23:23:26.538271+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:47:22.484108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032869134000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"24'19","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2026-02-22T23:23:26.500949+0000","last_change":"2026-02-22T23:23:22.760169+0000","last_active":"2026-02-22T23:23:26.500949+0000","last_peered":"2026-02-22T23:23:26.500949+0000","last_clean":"2026-02-22T23:23:26.500949+0000","last_became_active":"2026-02-22T23:23:22.740754+0000","last_became_peered":"2026-02-22T23:23:22.740754+0000","last_unstale":"2026-02-22T23:23:26.500949+0000","last_undegraded":"2026-02-22T23:23:26.500949+0000","last_fullsized":"2026-02-22T23:23:26.500949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_clean_scrub_stamp":"2026-02-22T23:23:20.759065+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:14:46.099842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":41,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":328192,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":28,"num_write":41,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":656384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,9,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2,4,5,6,7,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,3,4,5,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":6,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,4,5,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[2,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":294912,"data_stored":114834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[0,4,6,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,5,6,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":294912,"data_stored":114834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[2,4,6,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,5,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":622592,"data_stored":443007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":622592,"data_stored":443007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":290816,"data_stored":114815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":328192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":328192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:23:32.446 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:23:32.446 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-22T23:23:32.447 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-22T23:23:32.447 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:32.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.513+0000 7f189848f640 10 monclient: get_monmap_and_config 2026-02-22T23:23:32.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.513+0000 7f189848f640 10 monclient: build_initial_monmap 2026-02-22T23:23:32.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.513+0000 7f189848f640 10 monclient: monmap: 2026-02-22T23:23:32.516 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:32.516 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.516 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:32.515009+0000 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:32.515009+0000 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.513+0000 7f189848f640 1 Processor -- start 2026-02-22T23:23:32.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.513+0000 7f189848f640 1 -- start start 2026-02-22T23:23:32.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.513+0000 7f189848f640 10 monclient: init 2026-02-22T23:23:32.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:32.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient: _add_conns ranks=[5,0,2,1,4,3] 2026-02-22T23:23:32.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f189006ff20 0x7f1890070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient(hunting): picked mon.noname-c con 0x7f189006ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:32.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f18900c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:32.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f189015a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:32.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18900c7e90 con 0x7f18900c99a0 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18900c8170 con 0x7f189015a5e0 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1890159a40 con 0x7f189006ff20 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f189848f640 10 monclient(hunting): authenticate will time out at 1087.242377s 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f1895a03640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f18900c99a0 0x7f1890159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35620/0 (socket says 10.20.193.6:35620) 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f1895a03640 1 -- 10.20.193.6:0/1030966164 learned_addr learned my addr 10.20.193.6:0/1030966164 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.514+0000 7f1896204640 1 --2- 10.20.193.6:0/1030966164 >> v2:10.20.193.80:3300/0 conn(0x7f189006ff20 0x7f1890070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1896204640 10 monclient(hunting): get_auth_request con 0x7f189006ff20 auth_method 0 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1896204640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1896204640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1896204640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1896204640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1896204640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1896204640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1895202640 1 -- 10.20.193.6:0/1030966164 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625738055 0 0) 0x7f18900c7e90 con 0x7f18900c99a0 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1895202640 10 monclient(hunting): _negotiate 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1895202640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1895202640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1895202640 10 monclient(hunting): my global_id is 4593 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.515+0000 7f1895202640 1 -- 10.20.193.6:0/1030966164 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1878003620 con 0x7f18900c99a0 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 10 monclient(hunting): handle_auth_done global_id 4667 payload 274 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 1 -- 10.20.193.6:0/1030966164 >> v1:10.20.193.80:6789/0 conn(0x7f189015a5e0 legacy=0x7f189014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 1 -- 10.20.193.6:0/1030966164 >> v1:10.20.193.6:6789/0 conn(0x7f18900c99a0 legacy=0x7f1890159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 10 monclient: _finish_hunting 0 2026-02-22T23:23:32.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 1 monclient: found mon.noname-c 2026-02-22T23:23:32.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:32.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.516+0000 7f1896204640 1 -- 10.20.193.6:0/1030966164 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18900c8450 con 0x7f189006ff20 2026-02-22T23:23:32.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.518+0000 7f1896204640 1 --2- 10.20.193.6:0/1030966164 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f189006ff20 0x7f1890070300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18800098d0 tx=0x7f188002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eccf1ca86fa8ffe server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 1 -- 10.20.193.6:0/1030966164 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188003c070 con 0x7f189006ff20 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: dump: 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.530 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: _finish_auth 0 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: _check_auth_tickets 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 1 -- 10.20.193.6:0/1030966164 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188002fba0 con 0x7f189006ff20 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 5 monclient: authenticate success, global_id 4667 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 1 -- 10.20.193.6:0/1030966164 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880037440 con 0x7f189006ff20 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 4 set_mon_vals no callback set 2026-02-22T23:23:32.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f1895202640 10 monclient: dump: 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.532 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 10 monclient: shutdown 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 1 -- 10.20.193.6:0/1030966164 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f189006ff20 msgr2=0x7f1890070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 1 --2- 10.20.193.6:0/1030966164 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f189006ff20 0x7f1890070300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18800098d0 tx=0x7f188002eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 1 -- 10.20.193.6:0/1030966164 shutdown_connections 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 1 --2- 10.20.193.6:0/1030966164 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f189006ff20 0x7f1890070300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 1 -- 10.20.193.6:0/1030966164 >> 10.20.193.6:0/1030966164 conn(0x7f18900cd780 msgr2=0x7f18900cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 1 -- 10.20.193.6:0/1030966164 shutdown_connections 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.520+0000 7f189848f640 1 -- 10.20.193.6:0/1030966164 wait complete. 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient: build_initial_monmap 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient: monmap: 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:32.533 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 librados: starting msgr at 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 librados: starting objecter 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 Processor -- start 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 -- start start 2026-02-22T23:23:32.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 librados: setting wanted keys 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 librados: calling monclient init 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient: init 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f189006ff20 0x7f189014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): picked mon.noname-a con 0x7f189006ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f18900c99a0 0x7f18901553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): picked mon.noname-c con 0x7f18900c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f189015a5e0 0x7f189006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): picked mon.noname-b con 0x7f189015a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1890150520 con 0x7f189006ff20 2026-02-22T23:23:32.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f18900c8a10 con 0x7f189015a5e0 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18901506a0 con 0x7f18900c99a0 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.521+0000 7f189848f640 10 monclient(hunting): authenticate will time out at 1087.249825s 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f189006ff20 0x7f189014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f18900c99a0 0x7f18901553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896a05640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f189015a5e0 0x7f189006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f18900c99a0 0x7f18901553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:59076/0 (socket says 10.20.193.6:59076) 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 1 -- 10.20.193.6:0/2121210331 learned_addr learned my addr 10.20.193.6:0/2121210331 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 10 monclient(hunting): get_auth_request con 0x7f18900c99a0 auth_method 0 2026-02-22T23:23:32.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 10 monclient(hunting): get_auth_request con 0x7f189006ff20 auth_method 0 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896a05640 10 monclient(hunting): get_auth_request con 0x7f189015a5e0 auth_method 0 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896a05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896a05640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896a05640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1895a03640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.522+0000 7f1896204640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1896a05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1896a05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1896a05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 10 monclient(hunting): handle_auth_done global_id 4673 payload 824 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 1 -- 10.20.193.6:0/2121210331 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f189015a5e0 msgr2=0x7f189006e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f189015a5e0 0x7f189006e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 1 -- 10.20.193.6:0/2121210331 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f189006ff20 msgr2=0x7f189014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f189006ff20 0x7f189014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 10 monclient: _finish_hunting 0 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 1 monclient: found mon.noname-c 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.523+0000 7f1895a03640 1 -- 10.20.193.6:0/2121210331 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189014d1e0 con 0x7f18900c99a0 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.524+0000 7f1895a03640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f18900c99a0 0x7f18901553a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f188400c730 tx=0x7f188400cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.524+0000 7f1876ffd640 10 client.?.objecter ms_handle_connect 0x7f18900c99a0 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.524+0000 7f1876ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 1 -- 10.20.193.6:0/2121210331 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884017070 con 0x7f18900c99a0 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: dump: 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 5 monclient: authenticate success, global_id 4673 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 monclient: _renew_subs 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1890155de0 con 0x7f18900c99a0 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 client.4673.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 monclient: _renew_subs 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18901563a0 con 0x7f18900c99a0 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 1 librados: init done 2026-02-22T23:23:32.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 1 -- 10.20.193.6:0/2121210331 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1884005bd0 con 0x7f18900c99a0 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 1 -- 10.20.193.6:0/2121210331 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884021060 con 0x7f18900c99a0 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1894a01640 4 set_mon_vals no callback set 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1890070300 con 0x7f18900c99a0 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.526+0000 7f1876ffd640 10 monclient: dump: 2026-02-22T23:23:32.542 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:32.543 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:32.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.631+0000 7f1876ffd640 1 -- 10.20.193.6:0/2121210331 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1884024080 con 0x7f18900c99a0 2026-02-22T23:23:32.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.632+0000 7f1876ffd640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1860078e00 0x7f186007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:32.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.632+0000 7f1896204640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1860078e00 0x7f186007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:32.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.632+0000 7f1896204640 10 monclient: get_auth_request con 0x7f1860078e00 auth_method 0 2026-02-22T23:23:32.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.633+0000 7f1896204640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1860078e00 0x7f186007b2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1880039000 tx=0x7f1880033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:32.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.633+0000 7f1876ffd640 10 client.4673.objecter ms_handle_connect 0x7f1860078e00 2026-02-22T23:23:32.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.635+0000 7f1876ffd640 1 -- 10.20.193.6:0/2121210331 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 8972+0+0 (secure 0 0 0) 0x7f1884096e10 con 0x7f18900c99a0 2026-02-22T23:23:32.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.635+0000 7f1876ffd640 10 client.4673.objecter ms_dispatch2 0x7f1890000b90 osd_map(27..27 src has 1..27) 2026-02-22T23:23:32.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.635+0000 7f1876ffd640 3 client.4673.objecter handle_osd_map got epochs [27,27] > 0 2026-02-22T23:23:32.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.635+0000 7f1876ffd640 3 client.4673.objecter handle_osd_map decoding full epoch 27 2026-02-22T23:23:32.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.635+0000 7f1876ffd640 20 client.4673.objecter dump_active .. 0 homeless 2026-02-22T23:23:32.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.641+0000 7f1876ffd640 1 -- 10.20.193.6:0/2121210331 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f18840614d0 con 0x7f18900c99a0 2026-02-22T23:23:32.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.641+0000 7f1876ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:32.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.641+0000 7f1876ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:32.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.742+0000 7f189848f640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:32.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.742+0000 7f189848f640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:32.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.742+0000 7f189848f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:32.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.742+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1890159820 con 0x7f18900c99a0 2026-02-22T23:23:32.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.748+0000 7f1876ffd640 1 -- 10.20.193.6:0/2121210331 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f18840657d0 con 0x7f18900c99a0 2026-02-22T23:23:32.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.748+0000 7f1876ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:32.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.748+0000 7f1876ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:32.751 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:32.751 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 10 librados: watch_flush enter 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 10 librados: watch_flush exit 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 20 client.4673.objecter shutdown clearing up homeless session... 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 10 client.4673.objecter successfully canceled tick 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1860078e00 msgr2=0x7f186007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1860078e00 0x7f186007b2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1880039000 tx=0x7f1880033000 comp rx=0 tx=0).stop 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 10 monclient: shutdown 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f18900c99a0 msgr2=0x7f18901553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f18900c99a0 0x7f18901553a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f188400c730 tx=0x7f188400cc00 comp rx=0 tx=0).stop 2026-02-22T23:23:32.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 shutdown_connections 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1860078e00 0x7f186007b2c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f189015a5e0 0x7f189006e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f18900c99a0 0x7f18901553a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 --2- 10.20.193.6:0/2121210331 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f189006ff20 0x7f189014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 >> 10.20.193.6:0/2121210331 conn(0x7f18900cd780 msgr2=0x7f189014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 shutdown_connections 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 -- 10.20.193.6:0/2121210331 wait complete. 2026-02-22T23:23:32.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:32.749+0000 7f189848f640 1 librados: shutdown 2026-02-22T23:23:32.925 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:17.175 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:32.951 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:17.200 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:32.951 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:17.197 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:32.953 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:17.199 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:32.973 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:17.223 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:32.977 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:17.188 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:32.978 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:17.189 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:32.980 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:17.226 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:33.004 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:17.216 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:35.762 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:35.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.827+0000 7faddc0a0640 10 monclient: get_monmap_and_config 2026-02-22T23:23:35.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.827+0000 7faddc0a0640 10 monclient: build_initial_monmap 2026-02-22T23:23:35.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.827+0000 7faddc0a0640 10 monclient: monmap: 2026-02-22T23:23:35.830 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:35.829058+0000 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:35.829058+0000 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:35.831 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:35.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.827+0000 7faddc0a0640 1 Processor -- start 2026-02-22T23:23:35.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.827+0000 7faddc0a0640 1 -- start start 2026-02-22T23:23:35.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.827+0000 7faddc0a0640 10 monclient: init 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient: _add_conns ranks=[1,4,2,0,3,5] 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fadd406ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fadd40c99a0 0x7fadd4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient(hunting): picked mon.noname-b con 0x7fadd40c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fadd415a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadd40c7e90 con 0x7fadd406ff20 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadd40c8170 con 0x7fadd415a5e0 2026-02-22T23:23:35.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fadd4159a40 con 0x7fadd40c99a0 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7faddc0a0640 10 monclient(hunting): authenticate will time out at 1090.556414s 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7fadd9614640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fadd40c99a0 0x7fadd4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7fadda616640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fadd415a5e0 0x7fadd414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:36322/0 (socket says 10.20.193.6:36322) 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.828+0000 7fadda616640 1 -- 10.20.193.6:0/1928655750 learned_addr learned my addr 10.20.193.6:0/1928655750 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.829+0000 7fadd9614640 10 monclient(hunting): get_auth_request con 0x7fadd40c99a0 auth_method 0 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.829+0000 7fadd9614640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.829+0000 7fadd9614640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.829+0000 7fadd9614640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.829+0000 7fadd9614640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:35.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.829+0000 7fadd9614640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.829+0000 7fadd9614640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd8e13640 1 -- 10.20.193.6:0/1928655750 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742028535 0 0) 0x7fadd40c8170 con 0x7fadd415a5e0 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd8e13640 10 monclient(hunting): _negotiate 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd8e13640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd8e13640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd8e13640 10 monclient(hunting): my global_id is 4676 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd8e13640 1 -- 10.20.193.6:0/1928655750 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadbc003620 con 0x7fadd415a5e0 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 10 monclient(hunting): handle_auth_done global_id 4603 payload 274 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 1 -- 10.20.193.6:0/1928655750 >> v1:10.20.193.80:6789/0 conn(0x7fadd415a5e0 legacy=0x7fadd414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 1 -- 10.20.193.6:0/1928655750 >> v1:10.20.193.40:6789/0 conn(0x7fadd406ff20 legacy=0x7fadd4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 10 monclient: _finish_hunting 0 2026-02-22T23:23:35.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 1 monclient: found mon.noname-b 2026-02-22T23:23:35.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:35.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:35.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.830+0000 7fadd9614640 1 -- 10.20.193.6:0/1928655750 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd40c8450 con 0x7fadd40c99a0 2026-02-22T23:23:35.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.831+0000 7fadd9614640 1 --2- 10.20.193.6:0/1928655750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd40c99a0 0x7fadd4070300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadc80098d0 tx=0x7fadc802ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a58a82c3be2a65 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:35.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 1 -- 10.20.193.6:0/1928655750 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc803c070 con 0x7fadd40c99a0 2026-02-22T23:23:35.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:35.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: dump: 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: _finish_auth 0 2026-02-22T23:23:35.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: _check_auth_tickets 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 1 -- 10.20.193.6:0/1928655750 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc802fb00 con 0x7fadd40c99a0 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7faddc0a0640 5 monclient: authenticate success, global_id 4603 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7faddc0a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7faddc0a0640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 1 -- 10.20.193.6:0/1928655750 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8037440 con 0x7fadd40c99a0 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7faddc0a0640 4 set_mon_vals no callback set 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.834+0000 7fadd8e13640 10 monclient: dump: 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:35.844 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 10 monclient: shutdown 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 -- 10.20.193.6:0/1928655750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd40c99a0 msgr2=0x7fadd4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 --2- 10.20.193.6:0/1928655750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd40c99a0 0x7fadd4070300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fadc80098d0 tx=0x7fadc802ee80 comp rx=0 tx=0).stop 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 -- 10.20.193.6:0/1928655750 shutdown_connections 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 --2- 10.20.193.6:0/1928655750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd40c99a0 0x7fadd4070300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:35.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 -- 10.20.193.6:0/1928655750 >> 10.20.193.6:0/1928655750 conn(0x7fadd40cd780 msgr2=0x7fadd40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 -- 10.20.193.6:0/1928655750 shutdown_connections 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 -- 10.20.193.6:0/1928655750 wait complete. 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 10 monclient: build_initial_monmap 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 10 monclient: monmap: 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:35.846 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 librados: starting msgr at 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 librados: starting objecter 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 Processor -- start 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 -- start start 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 librados: setting wanted keys 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 1 librados: calling monclient init 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.835+0000 7faddc0a0640 10 monclient: init 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd406ff20 0x7fadd41f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): picked mon.noname-b con 0x7fadd406ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fadd40c99a0 0x7fadd41ed220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): picked mon.noname-a con 0x7fadd40c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:35.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fadd415a5e0 0x7fadd40c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): picked mon.noname-c con 0x7fadd415a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4150520 con 0x7fadd40c99a0 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fadd40c8a10 con 0x7fadd406ff20 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41506a0 con 0x7fadd415a5e0 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7faddc0a0640 10 monclient(hunting): authenticate will time out at 1090.564311s 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7fadda616640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fadd415a5e0 0x7fadd40c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.836+0000 7fadd9614640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fadd40c99a0 0x7fadd41ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fadd415a5e0 0x7fadd40c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:59090/0 (socket says 10.20.193.6:59090) 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 1 -- 10.20.193.6:0/2399221623 learned_addr learned my addr 10.20.193.6:0/2399221623 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 10 monclient(hunting): get_auth_request con 0x7fadd415a5e0 auth_method 0 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:35.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9614640 10 monclient(hunting): get_auth_request con 0x7fadd40c99a0 auth_method 0 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9614640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9614640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9614640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd406ff20 0x7fadd41f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 10 monclient(hunting): get_auth_request con 0x7fadd406ff20 auth_method 0 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9614640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9614640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9614640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:35.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadda616640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.837+0000 7fadd9e15640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 10 monclient(hunting): handle_auth_done global_id 4679 payload 824 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 1 -- 10.20.193.6:0/2399221623 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd406ff20 msgr2=0x7fadd41f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd406ff20 0x7fadd41f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 1 -- 10.20.193.6:0/2399221623 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fadd40c99a0 msgr2=0x7fadd41ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fadd40c99a0 0x7fadd41ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 10 monclient: _finish_hunting 0 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 1 monclient: found mon.noname-c 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.838+0000 7fadda616640 1 -- 10.20.193.6:0/2399221623 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd40c74c0 con 0x7fadd415a5e0 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.839+0000 7fadda616640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fadd415a5e0 0x7fadd40c1c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadd000caa0 tx=0x7fadd000cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:35.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.839+0000 7fadc27fc640 10 client.?.objecter ms_handle_connect 0x7fadd415a5e0 2026-02-22T23:23:35.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.839+0000 7fadc27fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 1 -- 10.20.193.6:0/2399221623 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd000ac40 con 0x7fadd415a5e0 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: dump: 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:35.852 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 1 -- 10.20.193.6:0/2399221623 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd000ade0 con 0x7fadd415a5e0 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 5 monclient: authenticate success, global_id 4679 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 10 monclient: _renew_subs 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd41f7790 con 0x7fadd415a5e0 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 1 -- 10.20.193.6:0/2399221623 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0012600 con 0x7fadd415a5e0 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc27fc640 10 monclient: dump: 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:35.853 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7fadc3fff640 4 set_mon_vals no callback set 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 10 client.4679.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 10 monclient: _renew_subs 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd4149220 con 0x7fadd415a5e0 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.841+0000 7faddc0a0640 1 librados: init done 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.842+0000 7faddc0a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.842+0000 7faddc0a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.842+0000 7faddc0a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:35.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.842+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd4070300 con 0x7fadd415a5e0 2026-02-22T23:23:35.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.853+0000 7fadc27fc640 1 -- 10.20.193.6:0/2399221623 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fadd00127a0 con 0x7fadd415a5e0 2026-02-22T23:23:35.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.853+0000 7fadc27fc640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fada4078e50 0x7fada407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:35.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.854+0000 7fadd9e15640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fada4078e50 0x7fada407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:35.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.854+0000 7fadd9e15640 10 monclient: get_auth_request con 0x7fada4078e50 auth_method 0 2026-02-22T23:23:35.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.854+0000 7fadd9e15640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fada4078e50 0x7fada407b310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fadc4005160 tx=0x7fadc4007f30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:35.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.854+0000 7fadc27fc640 10 client.4679.objecter ms_handle_connect 0x7fada4078e50 2026-02-22T23:23:35.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.855+0000 7fadc27fc640 1 -- 10.20.193.6:0/2399221623 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fadd005e830 con 0x7fadd415a5e0 2026-02-22T23:23:35.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.855+0000 7fadc27fc640 10 client.4679.objecter ms_dispatch2 0x7fadd4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:35.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.855+0000 7fadc27fc640 3 client.4679.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:35.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.855+0000 7fadc27fc640 3 client.4679.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:35.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.855+0000 7fadc27fc640 20 client.4679.objecter dump_active .. 0 homeless 2026-02-22T23:23:35.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.958+0000 7fadc27fc640 1 -- 10.20.193.6:0/2399221623 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fadd40c74c0 con 0x7fadd415a5e0 2026-02-22T23:23:35.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.958+0000 7fadc27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:35.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:35.958+0000 7fadc27fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:36.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.058+0000 7faddc0a0640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:36.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.058+0000 7faddc0a0640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:36.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.058+0000 7faddc0a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:36.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.058+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fadd40c4970 con 0x7fadd415a5e0 2026-02-22T23:23:36.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7fadc27fc640 1 -- 10.20.193.6:0/2399221623 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fadd00627f0 con 0x7fadd415a5e0 2026-02-22T23:23:36.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7fadc27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:36.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7fadc27fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:36.063 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:36.063 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 10 librados: watch_flush enter 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 10 librados: watch_flush exit 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 20 client.4679.objecter shutdown clearing up homeless session... 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 10 client.4679.objecter successfully canceled tick 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fada4078e50 msgr2=0x7fada407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fada4078e50 0x7fada407b310 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fadc4005160 tx=0x7fadc4007f30 comp rx=0 tx=0).stop 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 10 monclient: shutdown 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fadd415a5e0 msgr2=0x7fadd40c1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fadd415a5e0 0x7fadd40c1c50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadd000caa0 tx=0x7fadd000cf70 comp rx=0 tx=0).stop 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 shutdown_connections 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fada4078e50 0x7fada407b310 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fadd415a5e0 0x7fadd40c1c50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fadd40c99a0 0x7fadd41ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 --2- 10.20.193.6:0/2399221623 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fadd406ff20 0x7fadd41f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 >> 10.20.193.6:0/2399221623 conn(0x7fadd40cd780 msgr2=0x7fadd414e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.061+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 shutdown_connections 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.062+0000 7faddc0a0640 1 -- 10.20.193.6:0/2399221623 wait complete. 2026-02-22T23:23:36.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:36.062+0000 7faddc0a0640 1 librados: shutdown 2026-02-22T23:23:39.073 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.139+0000 7f60fd045640 10 monclient: get_monmap_and_config 2026-02-22T23:23:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.139+0000 7f60fd045640 10 monclient: build_initial_monmap 2026-02-22T23:23:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.139+0000 7f60fd045640 10 monclient: monmap: 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:39.141343+0000 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:39.141343+0000 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:39.143 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:39.144 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:39.144 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 1 Processor -- start 2026-02-22T23:23:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 1 -- start start 2026-02-22T23:23:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient: init 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient: _add_conns ranks=[0,1,5,4,3,2] 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f60f806ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f60f80c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f60f815a5e0 0x7f60f8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient(hunting): picked mon.noname-c con 0x7f60f815a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f80c7e90 con 0x7f60f806ff20 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f80c8170 con 0x7f60f80c99a0 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f60f8159a40 con 0x7f60f815a5e0 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.140+0000 7f60fd045640 10 monclient(hunting): authenticate will time out at 1093.868726s 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.141+0000 7f60f6d76640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f60f806ff20 0x7f60f8159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:59428/0 (socket says 10.20.193.6:59428) 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.141+0000 7f60f6d76640 1 -- 10.20.193.6:0/81996738 learned_addr learned my addr 10.20.193.6:0/81996738 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.141+0000 7f60f7577640 1 --2- 10.20.193.6:0/81996738 >> v2:10.20.193.80:3300/0 conn(0x7f60f815a5e0 0x7f60f8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.141+0000 7f60f7577640 10 monclient(hunting): get_auth_request con 0x7f60f815a5e0 auth_method 0 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.141+0000 7f60f7577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.141+0000 7f60f7577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.141+0000 7f60f7577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f7577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f7577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f7577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 1 -- 10.20.193.6:0/81996738 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3245933495 0 0) 0x7f60f80c7e90 con 0x7f60f806ff20 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): _negotiate 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): my global_id is 4602 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 1 -- 10.20.193.6:0/81996738 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60e4003620 con 0x7f60f806ff20 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 1 -- 10.20.193.6:0/81996738 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3363573412 0 0) 0x7f60f80c8170 con 0x7f60f80c99a0 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): _negotiate 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 10 monclient(hunting): my global_id is 4609 2026-02-22T23:23:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.142+0000 7f60f5d74640 1 -- 10.20.193.6:0/81996738 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60f80c7e90 con 0x7f60f80c99a0 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 10 monclient(hunting): handle_auth_done global_id 4682 payload 274 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 1 -- 10.20.193.6:0/81996738 >> v1:10.20.193.40:6789/0 conn(0x7f60f80c99a0 legacy=0x7f60f814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 1 -- 10.20.193.6:0/81996738 >> v1:10.20.193.6:6789/0 conn(0x7f60f806ff20 legacy=0x7f60f8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 10 monclient: _finish_hunting 0 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 1 monclient: found mon.noname-c 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.143+0000 7f60f7577640 1 -- 10.20.193.6:0/81996738 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f80c8450 con 0x7f60f815a5e0 2026-02-22T23:23:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.144+0000 7f60f7577640 1 --2- 10.20.193.6:0/81996738 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 0x7f60f8070300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009dc0 tx=0x7f60e802f280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4059eba49dbc533 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 1 -- 10.20.193.6:0/81996738 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e803c070 con 0x7f60f815a5e0 2026-02-22T23:23:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: dump: 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:39.158 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: _finish_auth 0 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: _check_auth_tickets 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 1 -- 10.20.193.6:0/81996738 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e8037440 con 0x7f60f815a5e0 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 1 -- 10.20.193.6:0/81996738 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8037810 con 0x7f60f815a5e0 2026-02-22T23:23:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60f5d74640 10 monclient: dump: 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:39.160 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 5 monclient: authenticate success, global_id 4682 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 4 set_mon_vals no callback set 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 10 monclient: shutdown 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 1 -- 10.20.193.6:0/81996738 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 msgr2=0x7f60f8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:39.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 1 --2- 10.20.193.6:0/81996738 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 0x7f60f8070300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009dc0 tx=0x7f60e802f280 comp rx=0 tx=0).stop 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 1 -- 10.20.193.6:0/81996738 shutdown_connections 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 1 --2- 10.20.193.6:0/81996738 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 0x7f60f8070300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 1 -- 10.20.193.6:0/81996738 >> 10.20.193.6:0/81996738 conn(0x7f60f80cd780 msgr2=0x7f60f80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 1 -- 10.20.193.6:0/81996738 shutdown_connections 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 1 -- 10.20.193.6:0/81996738 wait complete. 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 10 monclient: build_initial_monmap 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.146+0000 7f60fd045640 10 monclient: monmap: 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:39.162 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 librados: starting msgr at 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 librados: starting objecter 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 Processor -- start 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 -- start start 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 librados: setting wanted keys 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 librados: calling monclient init 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient: init 2026-02-22T23:23:39.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60f806ff20 0x7f60f81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): picked mon.noname-a con 0x7f60f806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60f80c99a0 0x7f60f81556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): picked mon.noname-b con 0x7f60f80c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 0x7f60f814c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): picked mon.noname-c con 0x7f60f815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f60f8150520 con 0x7f60f806ff20 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f60f80c8a10 con 0x7f60f80c99a0 2026-02-22T23:23:39.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f60f81506a0 con 0x7f60f815a5e0 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60fd045640 10 monclient(hunting): authenticate will time out at 1093.875610s 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60f6d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60f806ff20 0x7f60f81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.147+0000 7f60f7577640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 0x7f60f814c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60f806ff20 0x7f60f81551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:45758/0 (socket says 10.20.193.6:45758) 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 1 -- 10.20.193.6:0/2414246395 learned_addr learned my addr 10.20.193.6:0/2414246395 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 10 monclient(hunting): get_auth_request con 0x7f60f806ff20 auth_method 0 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f7577640 10 monclient(hunting): get_auth_request con 0x7f60f815a5e0 auth_method 0 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f7577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f7577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f7577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60f80c99a0 0x7f60f81556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 10 monclient(hunting): get_auth_request con 0x7f60f80c99a0 auth_method 0 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f7577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f7577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f7577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.148+0000 7f60f6575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 10 monclient(hunting): handle_auth_done global_id 4605 payload 824 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 1 -- 10.20.193.6:0/2414246395 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 msgr2=0x7f60f814c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:39.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 0x7f60f814c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 1 -- 10.20.193.6:0/2414246395 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60f80c99a0 msgr2=0x7f60f81556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60f80c99a0 0x7f60f81556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 10 monclient: _finish_hunting 0 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 1 monclient: found mon.noname-a 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.149+0000 7f60f6d76640 1 -- 10.20.193.6:0/2414246395 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f80c3f20 con 0x7f60f806ff20 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.151+0000 7f60f6d76640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60f806ff20 0x7f60f81551b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c760 tx=0x7f60ec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.151+0000 7f60d77fe640 10 client.?.objecter ms_handle_connect 0x7f60f806ff20 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.151+0000 7f60d77fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 1 -- 10.20.193.6:0/2414246395 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec017070 con 0x7f60f806ff20 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:39.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: dump: 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:39.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: _finish_auth 0 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 1 -- 10.20.193.6:0/2414246395 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ec005bd0 con 0x7f60f806ff20 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 1 -- 10.20.193.6:0/2414246395 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec007070 con 0x7f60f806ff20 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 5 monclient: authenticate success, global_id 4605 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60f5573640 4 set_mon_vals no callback set 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60d77fe640 10 monclient: dump: 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:39.170 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 10 monclient: _renew_subs 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f81543f0 con 0x7f60f806ff20 2026-02-22T23:23:39.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 10 client.4605.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 10 monclient: _renew_subs 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81549b0 con 0x7f60f806ff20 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.152+0000 7f60fd045640 1 librados: init done 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.153+0000 7f60fd045640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.153+0000 7f60fd045640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.153+0000 7f60fd045640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.153+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f8070300 con 0x7f60f806ff20 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.164+0000 7f60d77fe640 1 -- 10.20.193.6:0/2414246395 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f60ec007210 con 0x7f60f806ff20 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.165+0000 7f60d77fe640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60c4078e50 0x7f60c407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.165+0000 7f60f6575640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60c4078e50 0x7f60c407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.165+0000 7f60f6575640 10 monclient: get_auth_request con 0x7f60c4078e50 auth_method 0 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.165+0000 7f60f6575640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60c4078e50 0x7f60c407b310 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f60e0002410 tx=0x7f60e0003d50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.165+0000 7f60d77fe640 10 client.4605.objecter ms_handle_connect 0x7f60c4078e50 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.166+0000 7f60d77fe640 1 -- 10.20.193.6:0/2414246395 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f60ec05d500 con 0x7f60f806ff20 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.166+0000 7f60d77fe640 10 client.4605.objecter ms_dispatch2 0x7f60f8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.166+0000 7f60d77fe640 3 client.4605.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.166+0000 7f60d77fe640 3 client.4605.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.166+0000 7f60d77fe640 20 client.4605.objecter dump_active .. 0 homeless 2026-02-22T23:23:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.261+0000 7f60d77fe640 1 -- 10.20.193.6:0/2414246395 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f60ec021020 con 0x7f60f806ff20 2026-02-22T23:23:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.262+0000 7f60d77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.262+0000 7f60d77fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:39.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.361+0000 7f60fd045640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:39.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.361+0000 7f60fd045640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:39.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.361+0000 7f60fd045640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:39.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.361+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f60f81586a0 con 0x7f60f806ff20 2026-02-22T23:23:39.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.365+0000 7f60d77fe640 1 -- 10.20.193.6:0/2414246395 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f60ec0614c0 con 0x7f60f806ff20 2026-02-22T23:23:39.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.365+0000 7f60d77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:39.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.365+0000 7f60d77fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:39.368 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:39.368 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:39.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 10 librados: watch_flush enter 2026-02-22T23:23:39.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 10 librados: watch_flush exit 2026-02-22T23:23:39.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 20 client.4605.objecter shutdown clearing up homeless session... 2026-02-22T23:23:39.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 10 client.4605.objecter successfully canceled tick 2026-02-22T23:23:39.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60c4078e50 msgr2=0x7f60c407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:39.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60c4078e50 0x7f60c407b310 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f60e0002410 tx=0x7f60e0003d50 comp rx=0 tx=0).stop 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 10 monclient: shutdown 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60f806ff20 msgr2=0x7f60f81551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60f806ff20 0x7f60f81551b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00c760 tx=0x7f60ec00cc30 comp rx=0 tx=0).stop 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 shutdown_connections 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60c4078e50 0x7f60c407b310 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60f815a5e0 0x7f60f814c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60f80c99a0 0x7f60f81556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 --2- 10.20.193.6:0/2414246395 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60f806ff20 0x7f60f81551b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 >> 10.20.193.6:0/2414246395 conn(0x7f60f80cd780 msgr2=0x7f60f814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 shutdown_connections 2026-02-22T23:23:39.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 -- 10.20.193.6:0/2414246395 wait complete. 2026-02-22T23:23:39.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:39.367+0000 7f60fd045640 1 librados: shutdown 2026-02-22T23:23:42.379 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:42.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.444+0000 7f061c658640 10 monclient: get_monmap_and_config 2026-02-22T23:23:42.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.444+0000 7f061c658640 10 monclient: build_initial_monmap 2026-02-22T23:23:42.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.444+0000 7f061c658640 10 monclient: monmap: 2026-02-22T23:23:42.448 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:42.448 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:42.448 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:42.446578+0000 2026-02-22T23:23:42.448 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:42.446578+0000 2026-02-22T23:23:42.448 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 1 Processor -- start 2026-02-22T23:23:42.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 1 -- start start 2026-02-22T23:23:42.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient: init 2026-02-22T23:23:42.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:42.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient: _add_conns ranks=[5,2,1,0,3,4] 2026-02-22T23:23:42.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f061406ff20 0x7f0614070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:42.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient(hunting): picked mon.noname-c con 0x7f061406ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:42.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f06140c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:42.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f061415a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:42.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06140c7e90 con 0x7f061415a5e0 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06140c8170 con 0x7f06140c99a0 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0614159a40 con 0x7f061406ff20 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.445+0000 7f061c658640 10 monclient(hunting): authenticate will time out at 1097.173949s 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.446+0000 7f061abce640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f061415a5e0 0x7f061414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:45234/0 (socket says 10.20.193.6:45234) 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.446+0000 7f061abce640 1 -- 10.20.193.6:0/3342249398 learned_addr learned my addr 10.20.193.6:0/3342249398 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.446+0000 7f061a3cd640 1 --2- 10.20.193.6:0/3342249398 >> v2:10.20.193.80:3300/0 conn(0x7f061406ff20 0x7f0614070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.447+0000 7f061a3cd640 10 monclient(hunting): get_auth_request con 0x7f061406ff20 auth_method 0 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.447+0000 7f061a3cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.447+0000 7f061a3cd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:42.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.447+0000 7f061a3cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.447+0000 7f061a3cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.447+0000 7f061a3cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.447+0000 7f061a3cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.448+0000 7f06193cb640 1 -- 10.20.193.6:0/3342249398 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413810676 0 0) 0x7f06140c7e90 con 0x7f061415a5e0 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.448+0000 7f06193cb640 10 monclient(hunting): _negotiate 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.448+0000 7f06193cb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.448+0000 7f06193cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.448+0000 7f06193cb640 10 monclient(hunting): my global_id is 4615 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.448+0000 7f06193cb640 1 -- 10.20.193.6:0/3342249398 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05fc003620 con 0x7f061415a5e0 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 10 monclient(hunting): handle_auth_done global_id 4688 payload 274 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 1 -- 10.20.193.6:0/3342249398 >> v1:10.20.193.80:6789/0 conn(0x7f06140c99a0 legacy=0x7f0614159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 1 -- 10.20.193.6:0/3342249398 >> v1:10.20.193.40:6789/0 conn(0x7f061415a5e0 legacy=0x7f061414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:42.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:42.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 1 monclient: found mon.noname-c 2026-02-22T23:23:42.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:42.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:42.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.449+0000 7f061a3cd640 1 -- 10.20.193.6:0/3342249398 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06140c8450 con 0x7f061406ff20 2026-02-22T23:23:42.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.451+0000 7f061a3cd640 1 --2- 10.20.193.6:0/3342249398 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f061406ff20 0x7f0614070300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06040098d0 tx=0x7f060402eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c897505f6af62c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:42.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 1 -- 10.20.193.6:0/3342249398 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060403c070 con 0x7f061406ff20 2026-02-22T23:23:42.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:42.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:42.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: dump: 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: _finish_auth 0 2026-02-22T23:23:42.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 1 -- 10.20.193.6:0/3342249398 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f060402fba0 con 0x7f061406ff20 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 5 monclient: authenticate success, global_id 4688 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 1 -- 10.20.193.6:0/3342249398 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604037440 con 0x7f061406ff20 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f06193cb640 10 monclient: dump: 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:42.462 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 4 set_mon_vals no callback set 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 10 monclient: shutdown 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:42.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 1 -- 10.20.193.6:0/3342249398 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f061406ff20 msgr2=0x7f0614070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 1 --2- 10.20.193.6:0/3342249398 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f061406ff20 0x7f0614070300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f06040098d0 tx=0x7f060402eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 1 -- 10.20.193.6:0/3342249398 shutdown_connections 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 1 --2- 10.20.193.6:0/3342249398 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f061406ff20 0x7f0614070300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 1 -- 10.20.193.6:0/3342249398 >> 10.20.193.6:0/3342249398 conn(0x7f06140cd780 msgr2=0x7f06140cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 1 -- 10.20.193.6:0/3342249398 shutdown_connections 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.453+0000 7f061c658640 1 -- 10.20.193.6:0/3342249398 wait complete. 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 10 monclient: build_initial_monmap 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 10 monclient: monmap: 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:42.464 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 1 librados: starting msgr at 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 1 librados: starting objecter 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 1 Processor -- start 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 1 -- start start 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 1 librados: setting wanted keys 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 1 librados: calling monclient init 2026-02-22T23:23:42.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.454+0000 7f061c658640 10 monclient: init 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f061406ff20 0x7f06141f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): picked mon.noname-b con 0x7f061406ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f06140c99a0 0x7f0614156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): picked mon.noname-c con 0x7f06140c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f061415a5e0 0x7f06141f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): picked mon.noname-a con 0x7f061415a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:42.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0614150520 con 0x7f061415a5e0 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06140c8a10 con 0x7f061406ff20 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f06141506a0 con 0x7f06140c99a0 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061c658640 10 monclient(hunting): authenticate will time out at 1097.183366s 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061a3cd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f061406ff20 0x7f06141f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061abce640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f061415a5e0 0x7f06141f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f0619bcc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f06140c99a0 0x7f0614156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f0619bcc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f06140c99a0 0x7f0614156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43622/0 (socket says 10.20.193.6:43622) 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f0619bcc640 1 -- 10.20.193.6:0/1547767061 learned_addr learned my addr 10.20.193.6:0/1547767061 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f0619bcc640 10 monclient(hunting): get_auth_request con 0x7f06140c99a0 auth_method 0 2026-02-22T23:23:42.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f0619bcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f0619bcc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f0619bcc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061a3cd640 10 monclient(hunting): get_auth_request con 0x7f061406ff20 auth_method 0 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061a3cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061a3cd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.455+0000 7f061a3cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061abce640 10 monclient(hunting): get_auth_request con 0x7f061415a5e0 auth_method 0 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061abce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061abce640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061abce640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f0619bcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f0619bcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:42.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f0619bcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061a3cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061a3cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061a3cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061abce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061abce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.456+0000 7f061abce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 10 monclient(hunting): handle_auth_done global_id 4618 payload 824 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 1 -- 10.20.193.6:0/1547767061 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f06140c99a0 msgr2=0x7f0614156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f06140c99a0 0x7f0614156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 1 -- 10.20.193.6:0/1547767061 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f061415a5e0 msgr2=0x7f06141f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:42.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f061415a5e0 0x7f06141f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:42.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 1 monclient: found mon.noname-b 2026-02-22T23:23:42.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:42.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:42.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.457+0000 7f061a3cd640 1 -- 10.20.193.6:0/1547767061 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0614155440 con 0x7f061406ff20 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.458+0000 7f061a3cd640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f061406ff20 0x7f06141f64c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0614072430 tx=0x7f06040047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.458+0000 7f05faffd640 10 client.?.objecter ms_handle_connect 0x7f061406ff20 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.458+0000 7f05faffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 1 -- 10.20.193.6:0/1547767061 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060403c040 con 0x7f061406ff20 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: dump: 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.473 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 5 monclient: authenticate success, global_id 4618 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 monclient: _renew_subs 2026-02-22T23:23:42.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0614156cc0 con 0x7f061406ff20 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 client.4618.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 monclient: _renew_subs 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06141559c0 con 0x7f061406ff20 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 1 librados: init done 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 1 -- 10.20.193.6:0/1547767061 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f060402fa60 con 0x7f061406ff20 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f0618bca640 4 set_mon_vals no callback set 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 1 -- 10.20.193.6:0/1547767061 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604004220 con 0x7f061406ff20 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:42.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f05faffd640 10 monclient: dump: 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:42.476 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:42.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:42.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:42.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:42.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.460+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0614070300 con 0x7f061406ff20 2026-02-22T23:23:42.478 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.471+0000 7f05faffd640 1 -- 10.20.193.6:0/1547767061 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0604053070 con 0x7f061406ff20 2026-02-22T23:23:42.478 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.472+0000 7f05faffd640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05e4078e00 0x7f05e407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:42.478 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.472+0000 7f0619bcc640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05e4078e00 0x7f05e407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:42.478 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.472+0000 7f0619bcc640 10 monclient: get_auth_request con 0x7f05e4078e00 auth_method 0 2026-02-22T23:23:42.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.473+0000 7f0619bcc640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05e4078e00 0x7f05e407b2c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f061414cdc0 tx=0x7f0608002a30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:42.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.473+0000 7f05faffd640 10 client.4618.objecter ms_handle_connect 0x7f05e4078e00 2026-02-22T23:23:42.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.473+0000 7f05faffd640 1 -- 10.20.193.6:0/1547767061 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f06040bc970 con 0x7f061406ff20 2026-02-22T23:23:42.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.473+0000 7f05faffd640 10 client.4618.objecter ms_dispatch2 0x7f0614000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:42.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.473+0000 7f05faffd640 3 client.4618.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:42.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.473+0000 7f05faffd640 3 client.4618.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:42.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.473+0000 7f05faffd640 20 client.4618.objecter dump_active .. 0 homeless 2026-02-22T23:23:42.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.576+0000 7f05faffd640 1 -- 10.20.193.6:0/1547767061 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0604050020 con 0x7f061406ff20 2026-02-22T23:23:42.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.576+0000 7f05faffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:42.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.576+0000 7f05faffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:42.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.675+0000 7f061c658640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:42.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.675+0000 7f061c658640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:42.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.675+0000 7f061c658640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:42.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.675+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0614158180 con 0x7f061406ff20 2026-02-22T23:23:42.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.679+0000 7f05faffd640 1 -- 10.20.193.6:0/1547767061 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f06040871d0 con 0x7f061406ff20 2026-02-22T23:23:42.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.679+0000 7f05faffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:42.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.679+0000 7f05faffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:42.682 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:42.682 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 10 librados: watch_flush enter 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 10 librados: watch_flush exit 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 20 client.4618.objecter shutdown clearing up homeless session... 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 10 client.4618.objecter successfully canceled tick 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05e4078e00 msgr2=0x7f05e407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05e4078e00 0x7f05e407b2c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f061414cdc0 tx=0x7f0608002a30 comp rx=0 tx=0).stop 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 10 monclient: shutdown 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:42.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f061406ff20 msgr2=0x7f06141f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f061406ff20 0x7f06141f64c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0614072430 tx=0x7f06040047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 shutdown_connections 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05e4078e00 0x7f05e407b2c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f061415a5e0 0x7f06141f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f06140c99a0 0x7f0614156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 --2- 10.20.193.6:0/1547767061 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f061406ff20 0x7f06141f64c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 >> 10.20.193.6:0/1547767061 conn(0x7f06140cd780 msgr2=0x7f06140cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 shutdown_connections 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 -- 10.20.193.6:0/1547767061 wait complete. 2026-02-22T23:23:42.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:42.680+0000 7f061c658640 1 librados: shutdown 2026-02-22T23:23:45.691 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.757+0000 7f53a1885640 10 monclient: get_monmap_and_config 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.757+0000 7f53a1885640 10 monclient: build_initial_monmap 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.757+0000 7f53a1885640 10 monclient: monmap: 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:45.759050+0000 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:45.759050+0000 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.757+0000 7f53a1885640 1 Processor -- start 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.757+0000 7f53a1885640 1 -- start start 2026-02-22T23:23:45.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.757+0000 7f53a1885640 10 monclient: init 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient: _add_conns ranks=[4,3,0,5,1,2] 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f539c06ff20 0x7f539c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient(hunting): picked mon.noname-b con 0x7f539c06ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f539c0c99a0 0x7f539c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient(hunting): picked mon.noname-a con 0x7f539c0c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f539c15a410 addr v1:10.20.193.6:6789/0 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c15a860 con 0x7f539c15a410 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:45.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f539c159a40 con 0x7f539c0c99a0 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f539c0c7e90 con 0x7f539c06ff20 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f53a1885640 10 monclient(hunting): authenticate will time out at 1100.486445s 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.758+0000 7f539affd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f539c06ff20 0x7f539c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f539c06ff20 0x7f539c070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:40542/0 (socket says 10.20.193.6:40542) 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 1 -- 10.20.193.6:0/1256464400 learned_addr learned my addr 10.20.193.6:0/1256464400 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 10 monclient(hunting): get_auth_request con 0x7f539c06ff20 auth_method 0 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.759+0000 7f539affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 10 monclient(hunting): handle_auth_done global_id 4621 payload 274 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 1 -- 10.20.193.6:0/1256464400 >> v2:10.20.193.6:3300/0 conn(0x7f539c0c99a0 msgr2=0x7f539c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:45.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 1 --2- 10.20.193.6:0/1256464400 >> v2:10.20.193.6:3300/0 conn(0x7f539c0c99a0 0x7f539c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 1 -- 10.20.193.6:0/1256464400 >> v1:10.20.193.6:6789/0 conn(0x7f539c15a410 legacy=0x7f539c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:45.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:45.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 1 monclient: found mon.noname-b 2026-02-22T23:23:45.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:45.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:45.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.760+0000 7f539affd640 1 -- 10.20.193.6:0/1256464400 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0c8170 con 0x7f539c06ff20 2026-02-22T23:23:45.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.761+0000 7f539affd640 1 --2- 10.20.193.6:0/1256464400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c06ff20 0x7f539c070300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5390009970 tx=0x7f539002eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f0c79201bbbd30 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:45.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 1 -- 10.20.193.6:0/1256464400 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539003c070 con 0x7f539c06ff20 2026-02-22T23:23:45.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:45.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:45.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: dump: 2026-02-22T23:23:45.764 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f53a1885640 5 monclient: authenticate success, global_id 4621 2026-02-22T23:23:45.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f53a1885640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 1 -- 10.20.193.6:0/1256464400 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539002fb20 con 0x7f539c06ff20 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 1 -- 10.20.193.6:0/1256464400 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390037440 con 0x7f539c06ff20 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f5399ffb640 10 monclient: dump: 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:45.766 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f53a1885640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.762+0000 7f53a1885640 4 set_mon_vals no callback set 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 10 monclient: shutdown 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 -- 10.20.193.6:0/1256464400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c06ff20 msgr2=0x7f539c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 --2- 10.20.193.6:0/1256464400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c06ff20 0x7f539c070300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5390009970 tx=0x7f539002eef0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 -- 10.20.193.6:0/1256464400 shutdown_connections 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 --2- 10.20.193.6:0/1256464400 >> v2:10.20.193.6:3300/0 conn(0x7f539c0c99a0 0x7f539c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 --2- 10.20.193.6:0/1256464400 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c06ff20 0x7f539c070300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 -- 10.20.193.6:0/1256464400 >> 10.20.193.6:0/1256464400 conn(0x7f539c0cd780 msgr2=0x7f539c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 -- 10.20.193.6:0/1256464400 shutdown_connections 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 -- 10.20.193.6:0/1256464400 wait complete. 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 10 monclient: build_initial_monmap 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 10 monclient: monmap: 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:45.767 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 librados: starting msgr at 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 librados: starting objecter 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 Processor -- start 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 -- start start 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 librados: setting wanted keys 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 1 librados: calling monclient init 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.763+0000 7f53a1885640 10 monclient: init 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f539c06ff20 0x7f539c0c1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): picked mon.noname-c con 0x7f539c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c0c99a0 0x7f539c0c0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): picked mon.noname-b con 0x7f539c0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f539c15a410 0x7f539c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): picked mon.noname-a con 0x7f539c15a410 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0c8a10 con 0x7f539c15a410 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0c8730 con 0x7f539c0c99a0 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f539c150520 con 0x7f539c06ff20 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f53a1885640 10 monclient(hunting): authenticate will time out at 1100.492369s 2026-02-22T23:23:45.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539a7fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c0c99a0 0x7f539c0c0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:45.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539a7fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c0c99a0 0x7f539c0c0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:40552/0 (socket says 10.20.193.6:40552) 2026-02-22T23:23:45.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539a7fc640 1 -- 10.20.193.6:0/2598701308 learned_addr learned my addr 10.20.193.6:0/2598701308 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:45.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539a7fc640 10 monclient(hunting): get_auth_request con 0x7f539c0c99a0 auth_method 0 2026-02-22T23:23:45.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:45.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:45.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.764+0000 7f539affd640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f539c06ff20 0x7f539c0c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539affd640 10 monclient(hunting): get_auth_request con 0x7f539c06ff20 auth_method 0 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.765+0000 7f539affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 10 monclient(hunting): handle_auth_done global_id 4624 payload 824 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 1 -- 10.20.193.6:0/2598701308 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f539c06ff20 msgr2=0x7f539c0c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f539c06ff20 0x7f539c0c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 1 -- 10.20.193.6:0/2598701308 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f539c15a410 msgr2=0x7f539c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f539c15a410 0x7f539c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 1 monclient: found mon.noname-b 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.766+0000 7f539a7fc640 1 -- 10.20.193.6:0/2598701308 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0be4c0 con 0x7f539c0c99a0 2026-02-22T23:23:45.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.767+0000 7f539a7fc640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c0c99a0 0x7f539c0c0530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f538400b780 tx=0x7f538400bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:45.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.767+0000 7f5373fff640 10 client.?.objecter ms_handle_connect 0x7f539c0c99a0 2026-02-22T23:23:45.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.767+0000 7f5373fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 1 -- 10.20.193.6:0/2598701308 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384004480 con 0x7f539c0c99a0 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: dump: 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:45.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 1 -- 10.20.193.6:0/2598701308 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384010070 con 0x7f539c0c99a0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 1 -- 10.20.193.6:0/2598701308 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384002920 con 0x7f539c0c99a0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f5373fff640 10 monclient: dump: 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53997fa640 4 set_mon_vals no callback set 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 5 monclient: authenticate success, global_id 4624 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 10 monclient: _renew_subs 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c0be7a0 con 0x7f539c0c99a0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 10 client.4624.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 10 monclient: _renew_subs 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c0bed60 con 0x7f539c0c99a0 2026-02-22T23:23:45.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.769+0000 7f53a1885640 1 librados: init done 2026-02-22T23:23:45.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.770+0000 7f53a1885640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:45.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.770+0000 7f53a1885640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:45.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.770+0000 7f53a1885640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:45.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.770+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c070300 con 0x7f539c0c99a0 2026-02-22T23:23:45.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.781+0000 7f5373fff640 1 -- 10.20.193.6:0/2598701308 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5384002ac0 con 0x7f539c0c99a0 2026-02-22T23:23:45.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.782+0000 7f5373fff640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f536c078e00 0x7f536c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:45.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.782+0000 7f539affd640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f536c078e00 0x7f536c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:45.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.782+0000 7f539affd640 10 monclient: get_auth_request con 0x7f536c078e00 auth_method 0 2026-02-22T23:23:45.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.783+0000 7f5373fff640 10 client.4624.objecter ms_handle_connect 0x7f536c078e00 2026-02-22T23:23:45.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.783+0000 7f539affd640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f536c078e00 0x7f536c07b2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f539c0c5380 tx=0x7f5390033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:45.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.879+0000 7f5373fff640 1 -- 10.20.193.6:0/2598701308 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f538405d600 con 0x7f539c0c99a0 2026-02-22T23:23:45.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.879+0000 7f5373fff640 10 client.4624.objecter ms_dispatch2 0x7f539c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:45.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.879+0000 7f5373fff640 3 client.4624.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:45.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.879+0000 7f5373fff640 3 client.4624.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:45.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.879+0000 7f5373fff640 20 client.4624.objecter dump_active .. 0 homeless 2026-02-22T23:23:45.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.887+0000 7f5373fff640 1 -- 10.20.193.6:0/2598701308 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5384024080 con 0x7f539c0c99a0 2026-02-22T23:23:45.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.887+0000 7f5373fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:45.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.887+0000 7f5373fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:45.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.986+0000 7f53a1885640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:45.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.986+0000 7f53a1885640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:45.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.986+0000 7f53a1885640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:23:45.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.986+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f539c0cdc20 con 0x7f539c0c99a0 2026-02-22T23:23:45.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.989+0000 7f5373fff640 1 -- 10.20.193.6:0/2598701308 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5384061cc0 con 0x7f539c0c99a0 2026-02-22T23:23:45.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.989+0000 7f5373fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:45.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.989+0000 7f5373fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:45.991 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:45.991 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 10 librados: watch_flush enter 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 10 librados: watch_flush exit 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 20 client.4624.objecter shutdown clearing up homeless session... 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 10 client.4624.objecter successfully canceled tick 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f536c078e00 msgr2=0x7f536c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f536c078e00 0x7f536c07b2c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f539c0c5380 tx=0x7f5390033000 comp rx=0 tx=0).stop 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 10 monclient: shutdown 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c0c99a0 msgr2=0x7f539c0c0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c0c99a0 0x7f539c0c0530 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f538400b780 tx=0x7f538400bc50 comp rx=0 tx=0).stop 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 shutdown_connections 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f536c078e00 0x7f536c07b2c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f539c15a410 0x7f539c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f539c0c99a0 0x7f539c0c0530 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 --2- 10.20.193.6:0/2598701308 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f539c06ff20 0x7f539c0c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 >> 10.20.193.6:0/2598701308 conn(0x7f539c0cd780 msgr2=0x7f539c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 shutdown_connections 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 -- 10.20.193.6:0/2598701308 wait complete. 2026-02-22T23:23:45.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:45.990+0000 7f53a1885640 1 librados: shutdown 2026-02-22T23:23:49.002 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:49.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.069+0000 7fc148c1a640 10 monclient: get_monmap_and_config 2026-02-22T23:23:49.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.069+0000 7fc148c1a640 10 monclient: build_initial_monmap 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.069+0000 7fc148c1a640 10 monclient: monmap: 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:49.071275+0000 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:49.071275+0000 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:49.073 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:49.074 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:49.074 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:49.074 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:49.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 1 Processor -- start 2026-02-22T23:23:49.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 1 -- start start 2026-02-22T23:23:49.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient: init 2026-02-22T23:23:49.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:49.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient: _add_conns ranks=[5,1,2,4,0,3] 2026-02-22T23:23:49.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc14406ff20 0x7fc144070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:49.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient(hunting): picked mon.noname-c con 0x7fc14406ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:49.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc1440c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:49.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc14415a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440c7e90 con 0x7fc1440c99a0 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440c8170 con 0x7fc14415a5e0 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc144159a40 con 0x7fc14406ff20 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.070+0000 7fc148c1a640 10 monclient(hunting): authenticate will time out at 1103.798741s 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc14406ff20 0x7fc144070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc141d74640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fc1440c99a0 0x7fc144159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:52880/0 (socket says 10.20.193.6:52880) 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc141d74640 1 -- 10.20.193.6:0/2526758379 learned_addr learned my addr 10.20.193.6:0/2526758379 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 10 monclient(hunting): get_auth_request con 0x7fc14406ff20 auth_method 0 2026-02-22T23:23:49.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.071+0000 7fc142575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 10 monclient(hunting): handle_auth_done global_id 4700 payload 274 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 1 -- 10.20.193.6:0/2526758379 >> v1:10.20.193.80:6789/0 conn(0x7fc14415a5e0 legacy=0x7fc14414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 1 -- 10.20.193.6:0/2526758379 >> v1:10.20.193.40:6789/0 conn(0x7fc1440c99a0 legacy=0x7fc144159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 1 monclient: found mon.noname-c 2026-02-22T23:23:49.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc141573640 1 -- 10.20.193.6:0/2526758379 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3076267135 0 0) 0x7fc1440c7e90 con 0x7fc1440c99a0 2026-02-22T23:23:49.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:49.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:49.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc142575640 1 -- 10.20.193.6:0/2526758379 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1440c8450 con 0x7fc14406ff20 2026-02-22T23:23:49.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.072+0000 7fc141573640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.074+0000 7fc142575640 1 --2- 10.20.193.6:0/2526758379 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc144070300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098d0 tx=0x7fc13802eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be7a6aa5b413420 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 1 -- 10.20.193.6:0/2526758379 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13803c070 con 0x7fc14406ff20 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: dump: 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.086 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: _finish_auth 0 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: _check_auth_tickets 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 1 -- 10.20.193.6:0/2526758379 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13802fba0 con 0x7fc14406ff20 2026-02-22T23:23:49.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 1 -- 10.20.193.6:0/2526758379 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138037440 con 0x7fc14406ff20 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 5 monclient: authenticate success, global_id 4700 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 4 set_mon_vals no callback set 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc141573640 10 monclient: dump: 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:49.088 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 10 monclient: shutdown 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 1 -- 10.20.193.6:0/2526758379 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 msgr2=0x7fc144070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 1 --2- 10.20.193.6:0/2526758379 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc144070300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1380098d0 tx=0x7fc13802eed0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 1 -- 10.20.193.6:0/2526758379 shutdown_connections 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 1 --2- 10.20.193.6:0/2526758379 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc144070300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 1 -- 10.20.193.6:0/2526758379 >> 10.20.193.6:0/2526758379 conn(0x7fc1440cd780 msgr2=0x7fc1440cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:49.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 1 -- 10.20.193.6:0/2526758379 shutdown_connections 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 1 -- 10.20.193.6:0/2526758379 wait complete. 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 10 monclient: build_initial_monmap 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.076+0000 7fc148c1a640 10 monclient: monmap: 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:49.090 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 librados: starting msgr at 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 librados: starting objecter 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 Processor -- start 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 -- start start 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 librados: setting wanted keys 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 librados: calling monclient init 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient: init 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:49.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc14414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): picked mon.noname-c con 0x7fc14406ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc1440c99a0 0x7fc144155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): picked mon.noname-b con 0x7fc1440c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc14415a5e0 0x7fc144155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): picked mon.noname-a con 0x7fc14415a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc144150520 con 0x7fc14415a5e0 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440c8a10 con 0x7fc1440c99a0 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441506a0 con 0x7fc14406ff20 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:49.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc148c1a640 10 monclient(hunting): authenticate will time out at 1103.805559s 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc142575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc14414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc142d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc14415a5e0 0x7fc144155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.077+0000 7fc142575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc14414bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:52796/0 (socket says 10.20.193.6:52796) 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 1 -- 10.20.193.6:0/3734935878 learned_addr learned my addr 10.20.193.6:0/3734935878 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 10 monclient(hunting): get_auth_request con 0x7fc14406ff20 auth_method 0 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc1440c99a0 0x7fc144155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142d76640 10 monclient(hunting): get_auth_request con 0x7fc14415a5e0 auth_method 0 2026-02-22T23:23:49.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 10 monclient(hunting): get_auth_request con 0x7fc1440c99a0 auth_method 0 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc142d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:49.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.078+0000 7fc141d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 10 monclient(hunting): handle_auth_done global_id 4703 payload 824 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 1 -- 10.20.193.6:0/3734935878 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc1440c99a0 msgr2=0x7fc144155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc1440c99a0 0x7fc144155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 1 -- 10.20.193.6:0/3734935878 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc14415a5e0 msgr2=0x7fc144155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc14415a5e0 0x7fc144155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 10 monclient: _finish_hunting 0 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 1 monclient: found mon.noname-c 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:23:49.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.079+0000 7fc142575640 1 -- 10.20.193.6:0/3734935878 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14414c800 con 0x7fc14406ff20 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.080+0000 7fc142575640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc14414bc50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc138009130 tx=0x7fc1380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.080+0000 7fc12affd640 10 client.?.objecter ms_handle_connect 0x7fc14406ff20 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.080+0000 7fc12affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 1 -- 10.20.193.6:0/3734935878 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138046020 con 0x7fc14406ff20 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: dump: 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:49.096 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: _finish_auth 0 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 1 -- 10.20.193.6:0/3734935878 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138004240 con 0x7fc14406ff20 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 5 monclient: authenticate success, global_id 4703 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 1 -- 10.20.193.6:0/3734935878 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13803c040 con 0x7fc14406ff20 2026-02-22T23:23:49.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc12affd640 10 monclient: dump: 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:49.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc140d72640 4 set_mon_vals no callback set 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 monclient: _renew_subs 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14406ed80 con 0x7fc14406ff20 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 client.4703.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 monclient: _renew_subs 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc144154740 con 0x7fc14406ff20 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 1 librados: init done 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:49.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.082+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc144070300 con 0x7fc14406ff20 2026-02-22T23:23:49.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.184+0000 7fc12affd640 1 -- 10.20.193.6:0/3734935878 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc138053070 con 0x7fc14406ff20 2026-02-22T23:23:49.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.185+0000 7fc12affd640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc110078e30 0x7fc11007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:49.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.185+0000 7fc141d74640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc110078e30 0x7fc11007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:49.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.185+0000 7fc141d74640 10 monclient: get_auth_request con 0x7fc110078e30 auth_method 0 2026-02-22T23:23:49.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.185+0000 7fc141d74640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc110078e30 0x7fc11007b2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1440c0eb0 tx=0x7fc12c007df0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:49.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.185+0000 7fc12affd640 10 client.4703.objecter ms_handle_connect 0x7fc110078e30 2026-02-22T23:23:49.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.186+0000 7fc12affd640 1 -- 10.20.193.6:0/3734935878 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc138083600 con 0x7fc14406ff20 2026-02-22T23:23:49.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.186+0000 7fc12affd640 10 client.4703.objecter ms_dispatch2 0x7fc144000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:49.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.186+0000 7fc12affd640 3 client.4703.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:49.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.186+0000 7fc12affd640 3 client.4703.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:49.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.186+0000 7fc12affd640 20 client.4703.objecter dump_active .. 0 homeless 2026-02-22T23:23:49.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.193+0000 7fc12affd640 1 -- 10.20.193.6:0/3734935878 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc138050020 con 0x7fc14406ff20 2026-02-22T23:23:49.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.193+0000 7fc12affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:49.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.193+0000 7fc12affd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:49.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.295+0000 7fc148c1a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:49.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.295+0000 7fc148c1a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:49.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.295+0000 7fc148c1a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:23:49.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.295+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc144158280 con 0x7fc14406ff20 2026-02-22T23:23:49.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc12affd640 1 -- 10.20.193.6:0/3734935878 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc1380875c0 con 0x7fc14406ff20 2026-02-22T23:23:49.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc12affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:49.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc12affd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:49.299 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:49.299 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 10 librados: watch_flush enter 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 10 librados: watch_flush exit 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 20 client.4703.objecter shutdown clearing up homeless session... 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 10 client.4703.objecter successfully canceled tick 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc110078e30 msgr2=0x7fc11007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc110078e30 0x7fc11007b2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1440c0eb0 tx=0x7fc12c007df0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 10 monclient: shutdown 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 msgr2=0x7fc14414bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc14414bc50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc138009130 tx=0x7fc1380047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 shutdown_connections 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc110078e30 0x7fc11007b2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc14415a5e0 0x7fc144155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc1440c99a0 0x7fc144155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 --2- 10.20.193.6:0/3734935878 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc14406ff20 0x7fc14414bc50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 >> 10.20.193.6:0/3734935878 conn(0x7fc1440cd780 msgr2=0x7fc1440cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 shutdown_connections 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.298+0000 7fc148c1a640 1 -- 10.20.193.6:0/3734935878 wait complete. 2026-02-22T23:23:49.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:49.299+0000 7fc148c1a640 1 librados: shutdown 2026-02-22T23:23:52.311 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.377+0000 7efeeb92c640 10 monclient: get_monmap_and_config 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.377+0000 7efeeb92c640 10 monclient: build_initial_monmap 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.377+0000 7efeeb92c640 10 monclient: monmap: 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:52.377862+0000 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:52.377862+0000 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:52.378 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.377+0000 7efeeb92c640 1 Processor -- start 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.377+0000 7efeeb92c640 1 -- start start 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.377+0000 7efeeb92c640 10 monclient: init 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient: _add_conns ranks=[3,5,1,0,2,4] 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7efee41527d0 0x7efee406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient(hunting): picked mon.noname-a con 0x7efee41527d0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7efee4152bb0 0x7efee40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient(hunting): picked mon.noname-c con 0x7efee4152bb0 addr v2:10.20.193.80:3300/0 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efee40c5fc0 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee406e300 con 0x7efee40c5fc0 2026-02-22T23:23:52.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7efee406f1f0 con 0x7efee41527d0 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7efee40c6470 con 0x7efee4152bb0 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efeeb92c640 10 monclient(hunting): authenticate will time out at 1107.105317s 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee96a1640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7efee41527d0 0x7efee406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee8ea0640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7efee4152bb0 0x7efee40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee9ea2640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7efee40c5fc0 0x7efee40c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:52884/0 (socket says 10.20.193.6:52884) 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee9ea2640 1 -- 10.20.193.6:0/1716593374 learned_addr learned my addr 10.20.193.6:0/1716593374 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee96a1640 10 monclient(hunting): get_auth_request con 0x7efee41527d0 auth_method 0 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee96a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee96a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:52.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.378+0000 7efee96a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee8ea0640 10 monclient(hunting): get_auth_request con 0x7efee4152bb0 auth_method 0 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee8ea0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee8ea0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee8ea0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee96a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee96a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee96a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee8ea0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee8ea0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.379+0000 7efee8ea0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 10 monclient(hunting): handle_auth_done global_id 4614 payload 274 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 1 -- 10.20.193.6:0/1716593374 >> v2:10.20.193.80:3300/0 conn(0x7efee4152bb0 msgr2=0x7efee40c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 1 --2- 10.20.193.6:0/1716593374 >> v2:10.20.193.80:3300/0 conn(0x7efee4152bb0 0x7efee40c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 1 -- 10.20.193.6:0/1716593374 >> v1:10.20.193.40:6789/0 conn(0x7efee40c5fc0 legacy=0x7efee40c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 10 monclient: _finish_hunting 0 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 1 monclient: found mon.noname-a 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:52.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.380+0000 7efee96a1640 1 -- 10.20.193.6:0/1716593374 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee406e5e0 con 0x7efee41527d0 2026-02-22T23:23:52.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.381+0000 7efee96a1640 1 --2- 10.20.193.6:0/1716593374 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee41527d0 0x7efee406da60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efee40750e0 tx=0x7efed402ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806bd896cf03ee46 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:52.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 1 -- 10.20.193.6:0/1716593374 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed403c070 con 0x7efee41527d0 2026-02-22T23:23:52.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:52.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:52.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: dump: 2026-02-22T23:23:52.384 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 1 -- 10.20.193.6:0/1716593374 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed402faf0 con 0x7efee41527d0 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 5 monclient: authenticate success, global_id 4614 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 1 -- 10.20.193.6:0/1716593374 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4037440 con 0x7efee41527d0 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 4 set_mon_vals no callback set 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efed3fff640 10 monclient: dump: 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:52.385 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 10 monclient: shutdown 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 -- 10.20.193.6:0/1716593374 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee41527d0 msgr2=0x7efee406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1716593374 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee41527d0 0x7efee406da60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efee40750e0 tx=0x7efed402ede0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 -- 10.20.193.6:0/1716593374 shutdown_connections 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1716593374 >> v2:10.20.193.80:3300/0 conn(0x7efee4152bb0 0x7efee40c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1716593374 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee41527d0 0x7efee406da60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 -- 10.20.193.6:0/1716593374 >> 10.20.193.6:0/1716593374 conn(0x7efee40cd780 msgr2=0x7efee40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 -- 10.20.193.6:0/1716593374 shutdown_connections 2026-02-22T23:23:52.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.383+0000 7efeeb92c640 1 -- 10.20.193.6:0/1716593374 wait complete. 2026-02-22T23:23:52.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient: build_initial_monmap 2026-02-22T23:23:52.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient: monmap: 2026-02-22T23:23:52.387 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:52.387 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:52.387 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:52.387 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:52.387 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 librados: starting msgr at 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 librados: starting objecter 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 Processor -- start 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 -- start start 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 librados: setting wanted keys 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 librados: calling monclient init 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient: init 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee40c5fc0 0x7efee4148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): picked mon.noname-a con 0x7efee40c5fc0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efee41527d0 0x7efee40c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): picked mon.noname-c con 0x7efee41527d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efee4152bb0 0x7efee41f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): picked mon.noname-b con 0x7efee4152bb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7efee406ee80 con 0x7efee40c5fc0 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7efee406eba0 con 0x7efee4152bb0 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7efee406dfa0 con 0x7efee41527d0 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efeeb92c640 10 monclient(hunting): authenticate will time out at 1107.111716s 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.384+0000 7efee96a1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee40c5fc0 0x7efee4148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee40c5fc0 0x7efee4148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:51438/0 (socket says 10.20.193.6:51438) 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 1 -- 10.20.193.6:0/1992689877 learned_addr learned my addr 10.20.193.6:0/1992689877 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:52.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 10 monclient(hunting): get_auth_request con 0x7efee40c5fc0 auth_method 0 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efee41527d0 0x7efee40c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 10 monclient(hunting): get_auth_request con 0x7efee41527d0 auth_method 0 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efee4152bb0 0x7efee41f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee96a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 10 monclient(hunting): get_auth_request con 0x7efee4152bb0 auth_method 0 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee8ea0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:52.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.385+0000 7efee9ea2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 10 monclient(hunting): handle_auth_done global_id 4617 payload 824 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 1 -- 10.20.193.6:0/1992689877 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efee41527d0 msgr2=0x7efee40c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efee41527d0 0x7efee40c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 1 -- 10.20.193.6:0/1992689877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efee4152bb0 msgr2=0x7efee41f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efee4152bb0 0x7efee41f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 10 monclient: _finish_hunting 0 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 1 monclient: found mon.noname-a 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.386+0000 7efee96a1640 1 -- 10.20.193.6:0/1992689877 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee40c3970 con 0x7efee40c5fc0 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.387+0000 7efee96a1640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee40c5fc0 0x7efee4148c10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efed403a040 tx=0x7efed40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.387+0000 7efed1ffb640 10 client.?.objecter ms_handle_connect 0x7efee40c5fc0 2026-02-22T23:23:52.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.387+0000 7efed1ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 1 -- 10.20.193.6:0/1992689877 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed403c040 con 0x7efee40c5fc0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: dump: 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: _finish_auth 0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 1 -- 10.20.193.6:0/1992689877 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4042610 con 0x7efee40c5fc0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 5 monclient: authenticate success, global_id 4617 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 10 monclient: _renew_subs 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee40c3c50 con 0x7efee40c5fc0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 1 -- 10.20.193.6:0/1992689877 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed40429e0 con 0x7efee40c5fc0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed1ffb640 10 monclient: dump: 2026-02-22T23:23:52.391 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efed37fe640 4 set_mon_vals no callback set 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 10 client.4617.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 10 monclient: _renew_subs 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41f4b30 con 0x7efee40c5fc0 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.389+0000 7efeeb92c640 1 librados: init done 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.390+0000 7efeeb92c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.390+0000 7efeeb92c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.390+0000 7efeeb92c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:52.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.390+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee406da60 con 0x7efee40c5fc0 2026-02-22T23:23:52.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.401+0000 7efed1ffb640 1 -- 10.20.193.6:0/1992689877 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efed404b450 con 0x7efee40c5fc0 2026-02-22T23:23:52.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.401+0000 7efed1ffb640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efeb4078e80 0x7efeb407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:52.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.401+0000 7efee8ea0640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efeb4078e80 0x7efeb407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:52.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.401+0000 7efee8ea0640 10 monclient: get_auth_request con 0x7efeb4078e80 auth_method 0 2026-02-22T23:23:52.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.402+0000 7efed1ffb640 10 client.4617.objecter ms_handle_connect 0x7efeb4078e80 2026-02-22T23:23:52.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.402+0000 7efee8ea0640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efeb4078e80 0x7efeb407b340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efee40cc480 tx=0x7efed8009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:52.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.403+0000 7efed1ffb640 1 -- 10.20.193.6:0/1992689877 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7efed402fa70 con 0x7efee40c5fc0 2026-02-22T23:23:52.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.403+0000 7efed1ffb640 10 client.4617.objecter ms_dispatch2 0x7efee4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:52.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.403+0000 7efed1ffb640 3 client.4617.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:52.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.403+0000 7efed1ffb640 3 client.4617.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:52.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.403+0000 7efed1ffb640 20 client.4617.objecter dump_active .. 0 homeless 2026-02-22T23:23:52.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.498+0000 7efed1ffb640 1 -- 10.20.193.6:0/1992689877 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efed4086830 con 0x7efee40c5fc0 2026-02-22T23:23:52.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.498+0000 7efed1ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:52.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.498+0000 7efed1ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:52.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.600+0000 7efeeb92c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:52.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.600+0000 7efeeb92c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:52.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.600+0000 7efeeb92c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:52.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.600+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efee40cb5f0 con 0x7efee40c5fc0 2026-02-22T23:23:52.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.602+0000 7efed1ffb640 1 -- 10.20.193.6:0/1992689877 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7efed408de20 con 0x7efee40c5fc0 2026-02-22T23:23:52.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.602+0000 7efed1ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:52.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.602+0000 7efed1ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:52.604 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:52.604 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:52.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 10 librados: watch_flush enter 2026-02-22T23:23:52.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 10 librados: watch_flush exit 2026-02-22T23:23:52.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 20 client.4617.objecter shutdown clearing up homeless session... 2026-02-22T23:23:52.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 10 client.4617.objecter successfully canceled tick 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efeb4078e80 msgr2=0x7efeb407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efeb4078e80 0x7efeb407b340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efee40cc480 tx=0x7efed8009290 comp rx=0 tx=0).stop 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 10 monclient: shutdown 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee40c5fc0 msgr2=0x7efee4148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee40c5fc0 0x7efee4148c10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efed403a040 tx=0x7efed40047c0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 shutdown_connections 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efeb4078e80 0x7efeb407b340 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efee4152bb0 0x7efee41f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efee41527d0 0x7efee40c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 --2- 10.20.193.6:0/1992689877 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efee40c5fc0 0x7efee4148c10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:52.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.603+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 >> 10.20.193.6:0/1992689877 conn(0x7efee40cd780 msgr2=0x7efee40cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:52.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.604+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 shutdown_connections 2026-02-22T23:23:52.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.604+0000 7efeeb92c640 1 -- 10.20.193.6:0/1992689877 wait complete. 2026-02-22T23:23:52.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:52.604+0000 7efeeb92c640 1 librados: shutdown 2026-02-22T23:23:52.935 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:37.184 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.936 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:37.186 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.945 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:37.194 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.966 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:37.177 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.966 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:37.212 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.967 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:37.178 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.967 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:37.213 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.975 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:37.186 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:52.976 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:37.223 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:23:55.615 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:55.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.682+0000 7f00ee67c640 10 monclient: get_monmap_and_config 2026-02-22T23:23:55.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.682+0000 7f00ee67c640 10 monclient: build_initial_monmap 2026-02-22T23:23:55.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.682+0000 7f00ee67c640 10 monclient: monmap: 2026-02-22T23:23:55.684 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:55.684 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:55.684 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:55.683282+0000 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:55.683282+0000 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:55.685 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:55.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.683+0000 7f00ee67c640 1 Processor -- start 2026-02-22T23:23:55.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.683+0000 7f00ee67c640 1 -- start start 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.683+0000 7f00ee67c640 10 monclient: init 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.683+0000 7f00ee67c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient: _add_conns ranks=[4,2,3,0,5,1] 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f00e806ff20 0x7f00e8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient(hunting): picked mon.noname-b con 0x7f00e806ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f00e80c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f00e815a5e0 0x7f00e814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient(hunting): picked mon.noname-a con 0x7f00e815a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e80c7e90 con 0x7f00e80c99a0 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f00e8159a40 con 0x7f00e815a5e0 2026-02-22T23:23:55.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f00e80c8170 con 0x7f00e806ff20 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00ee67c640 10 monclient(hunting): authenticate will time out at 1110.411091s 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00e77fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f00e80c99a0 0x7f00e8159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:50182/0 (socket says 10.20.193.6:50182) 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00e77fe640 1 -- 10.20.193.6:0/3293531762 learned_addr learned my addr 10.20.193.6:0/3293531762 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00e7fff640 1 --2- 10.20.193.6:0/3293531762 >> v2:10.20.193.40:3300/0 conn(0x7f00e806ff20 0x7f00e8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00e7fff640 10 monclient(hunting): get_auth_request con 0x7f00e806ff20 auth_method 0 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.684+0000 7f00e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:55.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 1 --2- 10.20.193.6:0/3293531762 >> v2:10.20.193.6:3300/0 conn(0x7f00e815a5e0 0x7f00e814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e6ffd640 1 -- 10.20.193.6:0/3293531762 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649459961 0 0) 0x7f00e80c7e90 con 0x7f00e80c99a0 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e6ffd640 10 monclient(hunting): _negotiate 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e6ffd640 10 monclient(hunting): my global_id is 4712 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00e6ffd640 1 -- 10.20.193.6:0/3293531762 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00cc003620 con 0x7f00e80c99a0 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 10 monclient(hunting): get_auth_request con 0x7f00e815a5e0 auth_method 0 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:55.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.685+0000 7f00ecbf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 10 monclient(hunting): handle_auth_done global_id 4639 payload 274 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 1 -- 10.20.193.6:0/3293531762 >> v2:10.20.193.6:3300/0 conn(0x7f00e815a5e0 msgr2=0x7f00e814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 1 --2- 10.20.193.6:0/3293531762 >> v2:10.20.193.6:3300/0 conn(0x7f00e815a5e0 0x7f00e814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 1 -- 10.20.193.6:0/3293531762 >> v1:10.20.193.80:6789/0 conn(0x7f00e80c99a0 legacy=0x7f00e8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 1 monclient: found mon.noname-b 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.686+0000 7f00e7fff640 1 -- 10.20.193.6:0/3293531762 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80c8450 con 0x7f00e806ff20 2026-02-22T23:23:55.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.687+0000 7f00e7fff640 1 --2- 10.20.193.6:0/3293531762 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e806ff20 0x7f00e8070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00d40098a0 tx=0x7f00d402ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ebd4a461ff140c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:55.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 1 -- 10.20.193.6:0/3293531762 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d403c070 con 0x7f00e806ff20 2026-02-22T23:23:55.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:55.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:23:55.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: dump: 2026-02-22T23:23:55.701 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:55.701 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:55.701 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.701 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.701 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:55.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 1 -- 10.20.193.6:0/3293531762 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d402fae0 con 0x7f00e806ff20 2026-02-22T23:23:55.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:55.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 1 -- 10.20.193.6:0/3293531762 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4037440 con 0x7f00e806ff20 2026-02-22T23:23:55.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 5 monclient: authenticate success, global_id 4639 2026-02-22T23:23:55.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:55.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 4 set_mon_vals no callback set 2026-02-22T23:23:55.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:55.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00e6ffd640 10 monclient: dump: 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:55.704 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 10 monclient: shutdown 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 -- 10.20.193.6:0/3293531762 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e806ff20 msgr2=0x7f00e8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 --2- 10.20.193.6:0/3293531762 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e806ff20 0x7f00e8070300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f00d40098a0 tx=0x7f00d402ee10 comp rx=0 tx=0).stop 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 -- 10.20.193.6:0/3293531762 shutdown_connections 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 --2- 10.20.193.6:0/3293531762 >> v2:10.20.193.6:3300/0 conn(0x7f00e815a5e0 0x7f00e814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 --2- 10.20.193.6:0/3293531762 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e806ff20 0x7f00e8070300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 -- 10.20.193.6:0/3293531762 >> 10.20.193.6:0/3293531762 conn(0x7f00e80cd780 msgr2=0x7f00e80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 -- 10.20.193.6:0/3293531762 shutdown_connections 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.689+0000 7f00ee67c640 1 -- 10.20.193.6:0/3293531762 wait complete. 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient: build_initial_monmap 2026-02-22T23:23:55.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient: monmap: 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 librados: starting msgr at 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 librados: starting objecter 2026-02-22T23:23:55.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 Processor -- start 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 -- start start 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 librados: setting wanted keys 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 librados: calling monclient init 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient: init 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f00e806ff20 0x7f00e81f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): picked mon.noname-c con 0x7f00e806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e80c99a0 0x7f00e814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): picked mon.noname-b con 0x7f00e80c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f00e815a5e0 0x7f00e8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:55.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): picked mon.noname-a con 0x7f00e815a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8150350 con 0x7f00e815a5e0 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80c8a10 con 0x7f00e80c99a0 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81504d0 con 0x7f00e806ff20 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.690+0000 7f00ee67c640 10 monclient(hunting): authenticate will time out at 1110.417904s 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f00e815a5e0 0x7f00e8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f00e815a5e0 0x7f00e8157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:51460/0 (socket says 10.20.193.6:51460) 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 1 -- 10.20.193.6:0/302243110 learned_addr learned my addr 10.20.193.6:0/302243110 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:55.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 10 monclient(hunting): get_auth_request con 0x7f00e815a5e0 auth_method 0 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e80c99a0 0x7f00e814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f00e806ff20 0x7f00e81f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00ecbf2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 10 monclient(hunting): get_auth_request con 0x7f00e80c99a0 auth_method 0 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:55.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 10 monclient(hunting): get_auth_request con 0x7f00e806ff20 auth_method 0 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.691+0000 7f00e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 10 monclient(hunting): handle_auth_done global_id 4623 payload 824 2026-02-22T23:23:55.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 1 -- 10.20.193.6:0/302243110 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f00e806ff20 msgr2=0x7f00e81f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f00e806ff20 0x7f00e81f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 1 -- 10.20.193.6:0/302243110 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e80c99a0 msgr2=0x7f00e814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e80c99a0 0x7f00e814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 10 monclient: _finish_hunting 0 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 1 monclient: found mon.noname-a 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.692+0000 7f00ecbf2640 1 -- 10.20.193.6:0/302243110 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e8158840 con 0x7f00e815a5e0 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.693+0000 7f00ecbf2640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f00e815a5e0 0x7f00e8157840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009d30 tx=0x7f00dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.693+0000 7f00e4ff9640 10 client.?.objecter ms_handle_connect 0x7f00e815a5e0 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.693+0000 7f00e4ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 1 -- 10.20.193.6:0/302243110 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc018020 con 0x7f00e815a5e0 2026-02-22T23:23:55.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: dump: 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:55.712 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: _finish_auth 0 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 1 -- 10.20.193.6:0/302243110 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc014480 con 0x7f00e815a5e0 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 1 -- 10.20.193.6:0/302243110 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc014870 con 0x7f00e815a5e0 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 5 monclient: authenticate success, global_id 4623 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:55.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e67fc640 4 set_mon_vals no callback set 2026-02-22T23:23:55.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00e4ff9640 10 monclient: dump: 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:55.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 10 monclient: _renew_subs 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81e50a0 con 0x7f00e815a5e0 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 10 client.4623.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 10 monclient: _renew_subs 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e8158280 con 0x7f00e815a5e0 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.695+0000 7f00ee67c640 1 librados: init done 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.696+0000 7f00ee67c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.696+0000 7f00ee67c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.696+0000 7f00ee67c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:55.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.696+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8150860 con 0x7f00e815a5e0 2026-02-22T23:23:55.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.707+0000 7f00e4ff9640 1 -- 10.20.193.6:0/302243110 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f00dc014a10 con 0x7f00e815a5e0 2026-02-22T23:23:55.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.707+0000 7f00e4ff9640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f00b8078e30 0x7f00b807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:55.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.707+0000 7f00e7fff640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f00b8078e30 0x7f00b807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:55.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.707+0000 7f00e7fff640 10 monclient: get_auth_request con 0x7f00b8078e30 auth_method 0 2026-02-22T23:23:55.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.708+0000 7f00e7fff640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f00b8078e30 0x7f00b807b2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00d4039000 tx=0x7f00d4033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:55.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.708+0000 7f00e4ff9640 10 client.4623.objecter ms_handle_connect 0x7f00b8078e30 2026-02-22T23:23:55.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.709+0000 7f00e4ff9640 1 -- 10.20.193.6:0/302243110 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f00dc05ed50 con 0x7f00e815a5e0 2026-02-22T23:23:55.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.709+0000 7f00e4ff9640 10 client.4623.objecter ms_dispatch2 0x7f00e8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:55.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.709+0000 7f00e4ff9640 3 client.4623.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:55.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.709+0000 7f00e4ff9640 3 client.4623.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:55.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.709+0000 7f00e4ff9640 20 client.4623.objecter dump_active .. 0 homeless 2026-02-22T23:23:55.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.805+0000 7f00e4ff9640 1 -- 10.20.193.6:0/302243110 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f00dc022070 con 0x7f00e815a5e0 2026-02-22T23:23:55.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.805+0000 7f00e4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:55.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.805+0000 7f00e4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:55.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.907+0000 7f00ee67c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:55.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.907+0000 7f00ee67c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:55.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.907+0000 7f00ee67c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:55.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.907+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f00e81573f0 con 0x7f00e815a5e0 2026-02-22T23:23:55.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.910+0000 7f00e4ff9640 1 -- 10.20.193.6:0/302243110 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f00dc010040 con 0x7f00e815a5e0 2026-02-22T23:23:55.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.910+0000 7f00e4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:55.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.910+0000 7f00e4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:55.911 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:55.912 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:55.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 10 librados: watch_flush enter 2026-02-22T23:23:55.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 10 librados: watch_flush exit 2026-02-22T23:23:55.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 20 client.4623.objecter shutdown clearing up homeless session... 2026-02-22T23:23:55.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 10 client.4623.objecter successfully canceled tick 2026-02-22T23:23:55.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f00b8078e30 msgr2=0x7f00b807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f00b8078e30 0x7f00b807b2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f00d4039000 tx=0x7f00d4033000 comp rx=0 tx=0).stop 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 10 monclient: shutdown 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f00e815a5e0 msgr2=0x7f00e8157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f00e815a5e0 0x7f00e8157840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009d30 tx=0x7f00dc00c490 comp rx=0 tx=0).stop 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 shutdown_connections 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f00b8078e30 0x7f00b807b2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f00e815a5e0 0x7f00e8157840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f00e80c99a0 0x7f00e814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 --2- 10.20.193.6:0/302243110 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f00e806ff20 0x7f00e81f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 >> 10.20.193.6:0/302243110 conn(0x7f00e80cd780 msgr2=0x7f00e814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 shutdown_connections 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 -- 10.20.193.6:0/302243110 wait complete. 2026-02-22T23:23:55.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:55.911+0000 7f00ee67c640 1 librados: shutdown 2026-02-22T23:23:58.921 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.989+0000 7f021df11640 10 monclient: get_monmap_and_config 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.989+0000 7f021df11640 10 monclient: build_initial_monmap 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.989+0000 7f021df11640 10 monclient: monmap: 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:23:58.989885+0000 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:23:58.989885+0000 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:23:58.990 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.989+0000 7f021df11640 1 Processor -- start 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.989+0000 7f021df11640 1 -- start start 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.989+0000 7f021df11640 10 monclient: init 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient: _add_conns ranks=[1,3,2,5,0,4] 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f021806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f02180c99a0 0x7f0218070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient(hunting): picked mon.noname-a con 0x7f02180c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f021815a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02180c7e90 con 0x7f021806ff20 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02180c8170 con 0x7f021815a5e0 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0218159a40 con 0x7f02180c99a0 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:58.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f021df11640 10 monclient(hunting): authenticate will time out at 1113.717349s 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f0216ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f02180c99a0 0x7f0218070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f0216ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f02180c99a0 0x7f0218070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43430/0 (socket says 10.20.193.6:43430) 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f0216ffd640 1 -- 10.20.193.6:0/2312009934 learned_addr learned my addr 10.20.193.6:0/2312009934 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f0216ffd640 10 monclient(hunting): get_auth_request con 0x7f02180c99a0 auth_method 0 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f0216ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f0216ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.990+0000 7f0216ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.991+0000 7f0216ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.991+0000 7f0216ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:58.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.991+0000 7f0216ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 10 monclient(hunting): handle_auth_done global_id 4626 payload 274 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 1 -- 10.20.193.6:0/2312009934 >> v1:10.20.193.80:6789/0 conn(0x7f021815a5e0 legacy=0x7f021814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 1 -- 10.20.193.6:0/2312009934 >> v1:10.20.193.40:6789/0 conn(0x7f021806ff20 legacy=0x7f0218159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 1 monclient: found mon.noname-a 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 1 -- 10.20.193.6:0/2312009934 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02180c8450 con 0x7f02180c99a0 2026-02-22T23:23:58.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.992+0000 7f0216ffd640 1 --2- 10.20.193.6:0/2312009934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f02180c99a0 0x7f0218070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f02080099a0 tx=0x7f020802efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192433d34afc9114 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 1 -- 10.20.193.6:0/2312009934 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020803c070 con 0x7f02180c99a0 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: dump: 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: _finish_auth 0 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: _check_auth_tickets 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 1 -- 10.20.193.6:0/2312009934 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f020802fc70 con 0x7f02180c99a0 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 1 -- 10.20.193.6:0/2312009934 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0208037660 con 0x7f02180c99a0 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f02167fc640 10 monclient: dump: 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f021df11640 5 monclient: authenticate success, global_id 4626 2026-02-22T23:23:58.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f021df11640 10 monclient: get_monmap_and_config success 2026-02-22T23:23:58.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.994+0000 7f021df11640 4 set_mon_vals no callback set 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 10 monclient: shutdown 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 -- 10.20.193.6:0/2312009934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f02180c99a0 msgr2=0x7f0218070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 --2- 10.20.193.6:0/2312009934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f02180c99a0 0x7f0218070300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f02080099a0 tx=0x7f020802efa0 comp rx=0 tx=0).stop 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 -- 10.20.193.6:0/2312009934 shutdown_connections 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 --2- 10.20.193.6:0/2312009934 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f02180c99a0 0x7f0218070300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 -- 10.20.193.6:0/2312009934 >> 10.20.193.6:0/2312009934 conn(0x7f02180cd780 msgr2=0x7f02180cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 -- 10.20.193.6:0/2312009934 shutdown_connections 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 -- 10.20.193.6:0/2312009934 wait complete. 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 10 monclient: build_initial_monmap 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 10 monclient: monmap: 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 librados: starting msgr at 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 librados: starting objecter 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.995+0000 7f021df11640 1 Processor -- start 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 -- start start 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 librados: setting wanted keys 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 librados: calling monclient init 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient: init 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient: _reopen_session rank -1 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f021806ff20 0x7f02180c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): picked mon.noname-a con 0x7f021806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f02180c99a0 0x7f021814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): picked mon.noname-c con 0x7f02180c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f021815a5e0 0x7f02181f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): picked mon.noname-b con 0x7f021815a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:23:58.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0218150520 con 0x7f021806ff20 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f02180c8a10 con 0x7f021815a5e0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02181506a0 con 0x7f02180c99a0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): _renew_subs 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f021df11640 10 monclient(hunting): authenticate will time out at 1113.723424s 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f021806ff20 0x7f02180c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f021806ff20 0x7f02180c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43442/0 (socket says 10.20.193.6:43442) 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 1 -- 10.20.193.6:0/2943093645 learned_addr learned my addr 10.20.193.6:0/2943093645 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 10 monclient(hunting): get_auth_request con 0x7f021806ff20 auth_method 0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f0216ffd640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f02180c99a0 0x7f021814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f0217fff640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f021815a5e0 0x7f02181f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f0216ffd640 10 monclient(hunting): get_auth_request con 0x7f02180c99a0 auth_method 0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f0216ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f0216ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f0216ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.996+0000 7f02177fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0217fff640 10 monclient(hunting): get_auth_request con 0x7f021815a5e0 auth_method 0 2026-02-22T23:23:58.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0217fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0217fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0217fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0216ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0216ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0216ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0217fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0217fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.997+0000 7f0217fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 10 monclient(hunting): handle_auth_done global_id 4629 payload 824 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 1 -- 10.20.193.6:0/2943093645 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f02180c99a0 msgr2=0x7f021814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f02180c99a0 0x7f021814cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 1 -- 10.20.193.6:0/2943093645 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f021815a5e0 msgr2=0x7f02181f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f021815a5e0 0x7f02181f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 10 monclient: _finish_hunting 0 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 1 monclient: found mon.noname-a 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.998+0000 7f02177fe640 1 -- 10.20.193.6:0/2943093645 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02180c25a0 con 0x7f021806ff20 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.999+0000 7f02177fe640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f021806ff20 0x7f02180c3bc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f020400c580 tx=0x7f020400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.999+0000 7f01f3fff640 10 client.?.objecter ms_handle_connect 0x7f021806ff20 2026-02-22T23:23:59.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:58.999+0000 7f01f3fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.000+0000 7f01f3fff640 1 -- 10.20.193.6:0/2943093645 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204017070 con 0x7f021806ff20 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: dump: 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: _finish_auth 0 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 1 -- 10.20.193.6:0/2943093645 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02040052d0 con 0x7f021806ff20 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 1 -- 10.20.193.6:0/2943093645 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02040036b0 con 0x7f021806ff20 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 5 monclient: authenticate success, global_id 4629 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 monclient: _renew_subs 2026-02-22T23:23:59.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f0215ffb640 4 set_mon_vals no callback set 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02180c2840 con 0x7f021806ff20 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f01f3fff640 10 monclient: dump: 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 client.4629.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 monclient: _renew_subs 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02180c2e00 con 0x7f021806ff20 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 1 librados: init done 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:59.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.001+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0218070300 con 0x7f021806ff20 2026-02-22T23:23:59.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.012+0000 7f01f3fff640 1 -- 10.20.193.6:0/2943093645 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0204003850 con 0x7f021806ff20 2026-02-22T23:23:59.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.012+0000 7f01f3fff640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f01e8078e50 0x7f01e807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f0216ffd640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f01e8078e50 0x7f01e807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f0216ffd640 10 monclient: get_auth_request con 0x7f01e8078e50 auth_method 0 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f01f3fff640 1 -- 10.20.193.6:0/2943093645 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f020405d3e0 con 0x7f021806ff20 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f01f3fff640 10 client.4629.objecter ms_dispatch2 0x7f0218000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f01f3fff640 3 client.4629.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f01f3fff640 3 client.4629.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f0216ffd640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f01e8078e50 0x7f01e807b310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02180709c0 tx=0x7f0208033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f01f3fff640 20 client.4629.objecter dump_active .. 0 homeless 2026-02-22T23:23:59.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.013+0000 7f01f3fff640 10 client.4629.objecter ms_handle_connect 0x7f01e8078e50 2026-02-22T23:23:59.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.108+0000 7f01f3fff640 1 -- 10.20.193.6:0/2943093645 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0204021020 con 0x7f021806ff20 2026-02-22T23:23:59.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.108+0000 7f01f3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:23:59.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.108+0000 7f01f3fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:23:59.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.208+0000 7f021df11640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:23:59.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.208+0000 7f021df11640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:59.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.208+0000 7f021df11640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:23:59.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.208+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f021814f860 con 0x7f021806ff20 2026-02-22T23:23:59.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.211+0000 7f01f3fff640 1 -- 10.20.193.6:0/2943093645 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f02040613a0 con 0x7f021806ff20 2026-02-22T23:23:59.213 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:23:59.213 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:23:59.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.211+0000 7f01f3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.211+0000 7f01f3fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 10 librados: watch_flush enter 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 10 librados: watch_flush exit 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 20 client.4629.objecter shutdown clearing up homeless session... 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 10 client.4629.objecter successfully canceled tick 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f01e8078e50 msgr2=0x7f01e807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f01e8078e50 0x7f01e807b310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02180709c0 tx=0x7f0208033000 comp rx=0 tx=0).stop 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 10 monclient: shutdown 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:23:59.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f021806ff20 msgr2=0x7f02180c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f021806ff20 0x7f02180c3bc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f020400c580 tx=0x7f020400ca50 comp rx=0 tx=0).stop 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 shutdown_connections 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f01e8078e50 0x7f01e807b310 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f021815a5e0 0x7f02181f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f02180c99a0 0x7f021814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 --2- 10.20.193.6:0/2943093645 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f021806ff20 0x7f02180c3bc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 >> 10.20.193.6:0/2943093645 conn(0x7f02180cd780 msgr2=0x7f021806b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 shutdown_connections 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 -- 10.20.193.6:0/2943093645 wait complete. 2026-02-22T23:23:59.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:23:59.212+0000 7f021df11640 1 librados: shutdown 2026-02-22T23:24:02.224 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.288+0000 7f6e2cd11640 10 monclient: get_monmap_and_config 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.288+0000 7f6e2cd11640 10 monclient: build_initial_monmap 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.288+0000 7f6e2cd11640 10 monclient: monmap: 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:02.289509+0000 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:02.289509+0000 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:02.290 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 1 Processor -- start 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 1 -- start start 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient: init 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient: _add_conns ranks=[1,2,5,3,0,4] 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f6e2806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6e280c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6e2815a5e0 0x7f6e28070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient(hunting): picked mon.noname-c con 0x7f6e2815a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e280c7e90 con 0x7f6e2806ff20 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e280c8170 con 0x7f6e280c99a0 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6e28159a40 con 0x7f6e2815a5e0 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.289+0000 7f6e2cd11640 10 monclient(hunting): authenticate will time out at 1117.016994s 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.290+0000 7f6e26d76640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6e2815a5e0 0x7f6e28070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.290+0000 7f6e26575640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f6e2806ff20 0x7f6e28159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:54068/0 (socket says 10.20.193.6:54068) 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.290+0000 7f6e26575640 1 -- 10.20.193.6:0/856466944 learned_addr learned my addr 10.20.193.6:0/856466944 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.290+0000 7f6e26d76640 10 monclient(hunting): get_auth_request con 0x7f6e2815a5e0 auth_method 0 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.290+0000 7f6e26d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.290+0000 7f6e26d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:02.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.290+0000 7f6e26d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:02.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:02.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:02.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:02.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e25573640 1 -- 10.20.193.6:0/856466944 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445480808 0 0) 0x7f6e280c7e90 con 0x7f6e2806ff20 2026-02-22T23:24:02.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e25573640 10 monclient(hunting): _negotiate 2026-02-22T23:24:02.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e25573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:02.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e25573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:02.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.291+0000 7f6e25573640 10 monclient(hunting): my global_id is 4651 2026-02-22T23:24:02.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.292+0000 7f6e25573640 1 -- 10.20.193.6:0/856466944 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e14003620 con 0x7f6e2806ff20 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 10 monclient(hunting): handle_auth_done global_id 4724 payload 274 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 1 -- 10.20.193.6:0/856466944 >> v1:10.20.193.80:6789/0 conn(0x7f6e280c99a0 legacy=0x7f6e2814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 1 -- 10.20.193.6:0/856466944 >> v1:10.20.193.40:6789/0 conn(0x7f6e2806ff20 legacy=0x7f6e28159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 10 monclient: _finish_hunting 0 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 1 monclient: found mon.noname-c 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:02.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.293+0000 7f6e26d76640 1 -- 10.20.193.6:0/856466944 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e280c8450 con 0x7f6e2815a5e0 2026-02-22T23:24:02.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.295+0000 7f6e26d76640 1 --2- 10.20.193.6:0/856466944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e28070300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009dc0 tx=0x7f6e1802f280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=994b1479bb30c20d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 1 -- 10.20.193.6:0/856466944 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1803c070 con 0x7f6e2815a5e0 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: dump: 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:02.297 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: _finish_auth 0 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: _check_auth_tickets 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e2cd11640 5 monclient: authenticate success, global_id 4724 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e2cd11640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 1 -- 10.20.193.6:0/856466944 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18037440 con 0x7f6e2815a5e0 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 1 -- 10.20.193.6:0/856466944 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180377f0 con 0x7f6e2815a5e0 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e2cd11640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e2cd11640 4 set_mon_vals no callback set 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.296+0000 7f6e25573640 10 monclient: dump: 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:02.298 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 10 monclient: shutdown 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 -- 10.20.193.6:0/856466944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 msgr2=0x7f6e28070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/856466944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e28070300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009dc0 tx=0x7f6e1802f280 comp rx=0 tx=0).stop 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 -- 10.20.193.6:0/856466944 shutdown_connections 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/856466944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e28070300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 -- 10.20.193.6:0/856466944 >> 10.20.193.6:0/856466944 conn(0x7f6e280cd780 msgr2=0x7f6e280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 -- 10.20.193.6:0/856466944 shutdown_connections 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 -- 10.20.193.6:0/856466944 wait complete. 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 10 monclient: build_initial_monmap 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 10 monclient: monmap: 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:02.299 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 librados: starting msgr at 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 librados: starting objecter 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 Processor -- start 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 -- start start 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 librados: setting wanted keys 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 1 librados: calling monclient init 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.297+0000 7f6e2cd11640 10 monclient: init 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6e2806ff20 0x7f6e2806de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): picked mon.noname-b con 0x7f6e2806ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6e280c99a0 0x7f6e280bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): picked mon.noname-a con 0x7f6e280c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e280bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): picked mon.noname-c con 0x7f6e2815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6e28150520 con 0x7f6e280c99a0 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6e280c8a10 con 0x7f6e2806ff20 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6e281506a0 con 0x7f6e2815a5e0 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e2cd11640 10 monclient(hunting): authenticate will time out at 1117.025287s 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e25d74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6e280c99a0 0x7f6e280bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:02.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26d76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e280bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26575640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6e2806ff20 0x7f6e2806de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e25d74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6e280c99a0 0x7f6e280bf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43452/0 (socket says 10.20.193.6:43452) 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e25d74640 1 -- 10.20.193.6:0/3038496437 learned_addr learned my addr 10.20.193.6:0/3038496437 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e25d74640 10 monclient(hunting): get_auth_request con 0x7f6e280c99a0 auth_method 0 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e25d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e25d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e25d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26d76640 10 monclient(hunting): get_auth_request con 0x7f6e2815a5e0 auth_method 0 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26575640 10 monclient(hunting): get_auth_request con 0x7f6e2806ff20 auth_method 0 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.298+0000 7f6e26575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e25d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e25d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e25d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e26d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e26575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e26575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.299+0000 7f6e26575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 10 monclient(hunting): handle_auth_done global_id 4730 payload 824 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 1 -- 10.20.193.6:0/3038496437 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6e2806ff20 msgr2=0x7f6e2806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6e2806ff20 0x7f6e2806de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 1 -- 10.20.193.6:0/3038496437 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6e280c99a0 msgr2=0x7f6e280bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6e280c99a0 0x7f6e280bf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 10 monclient: _finish_hunting 0 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 1 monclient: found mon.noname-c 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.300+0000 7f6e26d76640 1 -- 10.20.193.6:0/3038496437 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2806e8a0 con 0x7f6e2815a5e0 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.301+0000 7f6e26d76640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e280bfe30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e18036000 tx=0x7f6e18004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.301+0000 7f6e0effd640 10 client.?.objecter ms_handle_connect 0x7f6e2815a5e0 2026-02-22T23:24:02.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.301+0000 7f6e0effd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 1 -- 10.20.193.6:0/3038496437 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1803c050 con 0x7f6e2815a5e0 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: dump: 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: _finish_auth 0 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: _check_auth_tickets 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 1 -- 10.20.193.6:0/3038496437 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18048070 con 0x7f6e2815a5e0 2026-02-22T23:24:02.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 1 -- 10.20.193.6:0/3038496437 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18004040 con 0x7f6e2815a5e0 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e24d72640 4 set_mon_vals no callback set 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e0effd640 10 monclient: dump: 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 5 monclient: authenticate success, global_id 4730 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 monclient: _renew_subs 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2806eb80 con 0x7f6e2815a5e0 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 client.4730.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 monclient: _renew_subs 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2806f140 con 0x7f6e2815a5e0 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 1 librados: init done 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:02.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.303+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e28070300 con 0x7f6e2815a5e0 2026-02-22T23:24:02.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.314+0000 7f6e0effd640 1 -- 10.20.193.6:0/3038496437 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6e180041e0 con 0x7f6e2815a5e0 2026-02-22T23:24:02.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.315+0000 7f6e0effd640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6df4078e30 0x7f6df407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:02.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.315+0000 7f6e26575640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6df4078e30 0x7f6df407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:02.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.315+0000 7f6e26575640 10 monclient: get_auth_request con 0x7f6df4078e30 auth_method 0 2026-02-22T23:24:02.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.315+0000 7f6e0effd640 10 client.4730.objecter ms_handle_connect 0x7f6df4078e30 2026-02-22T23:24:02.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.315+0000 7f6e26575640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6df4078e30 0x7f6df407b2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c007b60 tx=0x7f6e1c007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:02.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.316+0000 7f6e0effd640 1 -- 10.20.193.6:0/3038496437 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f6e18084100 con 0x7f6e2815a5e0 2026-02-22T23:24:02.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.316+0000 7f6e0effd640 10 client.4730.objecter ms_dispatch2 0x7f6e28000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:02.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.316+0000 7f6e0effd640 3 client.4730.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:02.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.316+0000 7f6e0effd640 3 client.4730.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:02.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.316+0000 7f6e0effd640 20 client.4730.objecter dump_active .. 0 homeless 2026-02-22T23:24:02.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.415+0000 7f6e0effd640 1 -- 10.20.193.6:0/3038496437 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6e180880c0 con 0x7f6e2815a5e0 2026-02-22T23:24:02.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.415+0000 7f6e0effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:02.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.415+0000 7f6e0effd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:02.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.514+0000 7f6e2cd11640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:02.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.514+0000 7f6e2cd11640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:02.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.514+0000 7f6e2cd11640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:02.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.514+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6e28158460 con 0x7f6e2815a5e0 2026-02-22T23:24:02.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.516+0000 7f6e0effd640 1 -- 10.20.193.6:0/3038496437 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6e1808c3c0 con 0x7f6e2815a5e0 2026-02-22T23:24:02.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.516+0000 7f6e0effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:02.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.516+0000 7f6e0effd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:02.518 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:02.518 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:02.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 10 librados: watch_flush enter 2026-02-22T23:24:02.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 10 librados: watch_flush exit 2026-02-22T23:24:02.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 20 client.4730.objecter shutdown clearing up homeless session... 2026-02-22T23:24:02.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 10 client.4730.objecter successfully canceled tick 2026-02-22T23:24:02.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6df4078e30 msgr2=0x7f6df407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6df4078e30 0x7f6df407b2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c007b60 tx=0x7f6e1c007ab0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 10 monclient: shutdown 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 msgr2=0x7f6e280bfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e280bfe30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e18036000 tx=0x7f6e18004b40 comp rx=0 tx=0).stop 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 shutdown_connections 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6df4078e30 0x7f6df407b2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6e2815a5e0 0x7f6e280bfe30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6e280c99a0 0x7f6e280bf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 --2- 10.20.193.6:0/3038496437 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6e2806ff20 0x7f6e2806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:02.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 >> 10.20.193.6:0/3038496437 conn(0x7f6e280cd780 msgr2=0x7f6e2814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:02.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 shutdown_connections 2026-02-22T23:24:02.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.517+0000 7f6e2cd11640 1 -- 10.20.193.6:0/3038496437 wait complete. 2026-02-22T23:24:02.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:02.518+0000 7f6e2cd11640 1 librados: shutdown 2026-02-22T23:24:05.528 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.593+0000 7f563c1c3640 10 monclient: get_monmap_and_config 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.593+0000 7f563c1c3640 10 monclient: build_initial_monmap 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.593+0000 7f563c1c3640 10 monclient: monmap: 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:05.594506+0000 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:05.594506+0000 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:05.595 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 1 Processor -- start 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 1 -- start start 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient: init 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient: _add_conns ranks=[0,2,4,3,5,1] 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f563406ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f56340c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f563415a5e0 0x7f5634070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient(hunting): picked mon.noname-b con 0x7f563415a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340c7e90 con 0x7f563406ff20 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340c8170 con 0x7f56340c99a0 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5634159a40 con 0x7f563415a5e0 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.594+0000 7f563c1c3640 10 monclient(hunting): authenticate will time out at 1120.321890s 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.595+0000 7f5639f38640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f563406ff20 0x7f5634159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35438/0 (socket says 10.20.193.6:35438) 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.595+0000 7f5639f38640 1 -- 10.20.193.6:0/3275091132 learned_addr learned my addr 10.20.193.6:0/3275091132 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.595+0000 7f563a739640 1 --2- 10.20.193.6:0/3275091132 >> v2:10.20.193.40:3300/0 conn(0x7f563415a5e0 0x7f5634070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.595+0000 7f563a739640 10 monclient(hunting): get_auth_request con 0x7f563415a5e0 auth_method 0 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.595+0000 7f563a739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.595+0000 7f563a739640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:05.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.595+0000 7f563a739640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 1 -- 10.20.193.6:0/3275091132 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 558618005 0 0) 0x7f56340c8170 con 0x7f56340c99a0 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): _negotiate 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): my global_id is 4733 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 1 -- 10.20.193.6:0/3275091132 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f561c003620 con 0x7f56340c99a0 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 1 -- 10.20.193.6:0/3275091132 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3888698682 0 0) 0x7f56340c7e90 con 0x7f563406ff20 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): _negotiate 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 10 monclient(hunting): my global_id is 4635 2026-02-22T23:24:05.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f5638f36640 1 -- 10.20.193.6:0/3275091132 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56340c8170 con 0x7f563406ff20 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 10 monclient(hunting): handle_auth_done global_id 4657 payload 274 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 1 -- 10.20.193.6:0/3275091132 >> v1:10.20.193.80:6789/0 conn(0x7f56340c99a0 legacy=0x7f563414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 1 -- 10.20.193.6:0/3275091132 >> v1:10.20.193.6:6789/0 conn(0x7f563406ff20 legacy=0x7f5634159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 10 monclient: _finish_hunting 0 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 1 monclient: found mon.noname-b 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.596+0000 7f563a739640 1 -- 10.20.193.6:0/3275091132 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340c8450 con 0x7f563415a5e0 2026-02-22T23:24:05.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.597+0000 7f563a739640 1 --2- 10.20.193.6:0/3275091132 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f563415a5e0 0x7f5634070300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5630009dc0 tx=0x7f563002f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba3e5a8bbe7f6fc server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 1 -- 10.20.193.6:0/3275091132 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563003c070 con 0x7f563415a5e0 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: dump: 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.600 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: _finish_auth 0 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: _check_auth_tickets 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 1 -- 10.20.193.6:0/3275091132 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5630037440 con 0x7f563415a5e0 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 5 monclient: authenticate success, global_id 4657 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 1 -- 10.20.193.6:0/3275091132 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630037810 con 0x7f563415a5e0 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 4 set_mon_vals no callback set 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f5638f36640 10 monclient: dump: 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:05.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 10 monclient: shutdown 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 1 -- 10.20.193.6:0/3275091132 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f563415a5e0 msgr2=0x7f5634070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3275091132 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f563415a5e0 0x7f5634070300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5630009dc0 tx=0x7f563002f280 comp rx=0 tx=0).stop 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 1 -- 10.20.193.6:0/3275091132 shutdown_connections 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3275091132 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f563415a5e0 0x7f5634070300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 1 -- 10.20.193.6:0/3275091132 >> 10.20.193.6:0/3275091132 conn(0x7f56340cd780 msgr2=0x7f56340cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.599+0000 7f563c1c3640 1 -- 10.20.193.6:0/3275091132 shutdown_connections 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 1 -- 10.20.193.6:0/3275091132 wait complete. 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 10 monclient: build_initial_monmap 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 10 monclient: monmap: 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 1 librados: starting msgr at 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 1 librados: starting objecter 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 1 Processor -- start 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 1 -- start start 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 1 librados: setting wanted keys 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 1 librados: calling monclient init 2026-02-22T23:24:05.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.600+0000 7f563c1c3640 10 monclient: init 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f563406ff20 0x7f563414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): picked mon.noname-a con 0x7f563406ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f56340c99a0 0x7f56340bec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): picked mon.noname-b con 0x7f56340c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f563415a5e0 0x7f56340bf190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): picked mon.noname-c con 0x7f563415a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5634150520 con 0x7f563406ff20 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f56340c8a10 con 0x7f56340c99a0 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f56341506a0 con 0x7f563415a5e0 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563c1c3640 10 monclient(hunting): authenticate will time out at 1120.328309s 2026-02-22T23:24:05.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f563406ff20 0x7f563414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639737640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f56340c99a0 0x7f56340bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f563406ff20 0x7f563414c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43466/0 (socket says 10.20.193.6:43466) 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 1 -- 10.20.193.6:0/3080525754 learned_addr learned my addr 10.20.193.6:0/3080525754 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 10 monclient(hunting): get_auth_request con 0x7f563406ff20 auth_method 0 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639737640 10 monclient(hunting): get_auth_request con 0x7f56340c99a0 auth_method 0 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639737640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639737640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639737640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f5639f38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.601+0000 7f563a739640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f563415a5e0 0x7f56340bf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f5639737640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f5639737640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f5639737640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f563a739640 10 monclient(hunting): get_auth_request con 0x7f563415a5e0 auth_method 0 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f563a739640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f563a739640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:05.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f563a739640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f563a739640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f563a739640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.602+0000 7f563a739640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 10 monclient(hunting): handle_auth_done global_id 4638 payload 824 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 1 -- 10.20.193.6:0/3080525754 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f563415a5e0 msgr2=0x7f56340bf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f563415a5e0 0x7f56340bf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 1 -- 10.20.193.6:0/3080525754 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f56340c99a0 msgr2=0x7f56340bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f56340c99a0 0x7f56340bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 10 monclient: _finish_hunting 0 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 1 monclient: found mon.noname-a 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.603+0000 7f5639f38640 1 -- 10.20.193.6:0/3080525754 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340bdbb0 con 0x7f563406ff20 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.604+0000 7f5639f38640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f563406ff20 0x7f563414c670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f562400c760 tx=0x7f562400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.604+0000 7f56227fc640 10 client.?.objecter ms_handle_connect 0x7f563406ff20 2026-02-22T23:24:05.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.604+0000 7f56227fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 1 -- 10.20.193.6:0/3080525754 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624017070 con 0x7f563406ff20 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: dump: 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: _finish_auth 0 2026-02-22T23:24:05.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: _check_auth_tickets 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 1 -- 10.20.193.6:0/3080525754 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5624005bb0 con 0x7f563406ff20 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 5 monclient: authenticate success, global_id 4638 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 1 -- 10.20.193.6:0/3080525754 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624007070 con 0x7f563406ff20 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f5623fff640 4 set_mon_vals no callback set 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f56227fc640 10 monclient: dump: 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 monclient: _renew_subs 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56340bde90 con 0x7f563406ff20 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 client.4638.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 monclient: _renew_subs 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56340be450 con 0x7f563406ff20 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 1 librados: init done 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:05.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:05.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.606+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634070300 con 0x7f563406ff20 2026-02-22T23:24:05.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.621+0000 7f56227fc640 1 -- 10.20.193.6:0/3080525754 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5624007210 con 0x7f563406ff20 2026-02-22T23:24:05.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.621+0000 7f56227fc640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5604078e80 0x7f560407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:05.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.621+0000 7f5639737640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5604078e80 0x7f560407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:05.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.621+0000 7f5639737640 10 monclient: get_auth_request con 0x7f5604078e80 auth_method 0 2026-02-22T23:24:05.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.622+0000 7f56227fc640 10 client.4638.objecter ms_handle_connect 0x7f5604078e80 2026-02-22T23:24:05.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.622+0000 7f5639737640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5604078e80 0x7f560407b340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56341571c0 tx=0x7f5628003d70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:05.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.627+0000 7f56227fc640 1 -- 10.20.193.6:0/3080525754 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f562405d440 con 0x7f563406ff20 2026-02-22T23:24:05.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.627+0000 7f56227fc640 10 client.4638.objecter ms_dispatch2 0x7f5634000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:05.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.627+0000 7f56227fc640 3 client.4638.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:05.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.627+0000 7f56227fc640 3 client.4638.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:05.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.627+0000 7f56227fc640 20 client.4638.objecter dump_active .. 0 homeless 2026-02-22T23:24:05.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.719+0000 7f56227fc640 1 -- 10.20.193.6:0/3080525754 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5624021020 con 0x7f563406ff20 2026-02-22T23:24:05.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.719+0000 7f56227fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:05.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.719+0000 7f56227fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:05.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.825+0000 7f563c1c3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:05.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.825+0000 7f563c1c3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:05.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.825+0000 7f563c1c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:05.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.825+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f56341573f0 con 0x7f563406ff20 2026-02-22T23:24:05.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.827+0000 7f56227fc640 1 -- 10.20.193.6:0/3080525754 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5624061400 con 0x7f563406ff20 2026-02-22T23:24:05.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.827+0000 7f56227fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:05.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.827+0000 7f56227fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:05.829 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:05.829 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:05.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 10 librados: watch_flush enter 2026-02-22T23:24:05.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 10 librados: watch_flush exit 2026-02-22T23:24:05.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 20 client.4638.objecter shutdown clearing up homeless session... 2026-02-22T23:24:05.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 10 client.4638.objecter successfully canceled tick 2026-02-22T23:24:05.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5604078e80 msgr2=0x7f560407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5604078e80 0x7f560407b340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f56341571c0 tx=0x7f5628003d70 comp rx=0 tx=0).stop 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 10 monclient: shutdown 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f563406ff20 msgr2=0x7f563414c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f563406ff20 0x7f563414c670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f562400c760 tx=0x7f562400cc30 comp rx=0 tx=0).stop 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 shutdown_connections 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5604078e80 0x7f560407b340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f563415a5e0 0x7f56340bf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f56340c99a0 0x7f56340bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 --2- 10.20.193.6:0/3080525754 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f563406ff20 0x7f563414c670 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 >> 10.20.193.6:0/3080525754 conn(0x7f56340cd780 msgr2=0x7f56340cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:05.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 shutdown_connections 2026-02-22T23:24:05.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 -- 10.20.193.6:0/3080525754 wait complete. 2026-02-22T23:24:05.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:05.828+0000 7f563c1c3640 1 librados: shutdown 2026-02-22T23:24:08.838 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:08.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.903+0000 7ff282032640 10 monclient: get_monmap_and_config 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.903+0000 7ff282032640 10 monclient: build_initial_monmap 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.903+0000 7ff282032640 10 monclient: monmap: 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:08.903829+0000 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:08.903829+0000 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:08.905 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:08.906 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:08.906 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:08.906 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:08.906 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:08.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.903+0000 7ff282032640 1 Processor -- start 2026-02-22T23:24:08.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.903+0000 7ff282032640 1 -- start start 2026-02-22T23:24:08.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.903+0000 7ff282032640 10 monclient: init 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient: _add_conns ranks=[0,3,2,1,4,5] 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff27c1527d0 addr v1:10.20.193.6:6789/0 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff27c152bb0 0x7ff27c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient(hunting): picked mon.noname-a con 0x7ff27c152bb0 addr v2:10.20.193.6:3300/0 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff27c0c5fc0 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27c06e300 con 0x7ff27c1527d0 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27c06e5e0 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff27c06f1f0 con 0x7ff27c152bb0 2026-02-22T23:24:08.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff282032640 10 monclient(hunting): authenticate will time out at 1123.631283s 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff27affd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff27c152bb0 0x7ff27c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff27b7fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7ff27c1527d0 0x7ff27c06da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:53624/0 (socket says 10.20.193.6:53624) 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff27b7fe640 1 -- 10.20.193.6:0/534083144 learned_addr learned my addr 10.20.193.6:0/534083144 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff27affd640 10 monclient(hunting): get_auth_request con 0x7ff27c152bb0 auth_method 0 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff27affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff27affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.904+0000 7ff27affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.905+0000 7ff27affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.905+0000 7ff27affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.905+0000 7ff27affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27a7fc640 1 -- 10.20.193.6:0/534083144 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672388949 0 0) 0x7ff27c06e5e0 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27a7fc640 10 monclient(hunting): my global_id is 4739 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27a7fc640 1 -- 10.20.193.6:0/534083144 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff260003620 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 10 monclient(hunting): handle_auth_done global_id 4641 payload 274 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 1 -- 10.20.193.6:0/534083144 >> v1:10.20.193.80:6789/0 conn(0x7ff27c0c5fc0 legacy=0x7ff27c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 1 -- 10.20.193.6:0/534083144 >> v1:10.20.193.6:6789/0 conn(0x7ff27c1527d0 legacy=0x7ff27c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 10 monclient: _finish_hunting 0 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 1 monclient: found mon.noname-a 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:08.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.906+0000 7ff27affd640 1 -- 10.20.193.6:0/534083144 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c06e8c0 con 0x7ff27c152bb0 2026-02-22T23:24:08.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.908+0000 7ff27affd640 1 --2- 10.20.193.6:0/534083144 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c152bb0 0x7ff27c0c5a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0098a0 tx=0x7ff26c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e17dbd3a864d0de server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:08.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 1 -- 10.20.193.6:0/534083144 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c03c070 con 0x7ff27c152bb0 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: dump: 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 1 -- 10.20.193.6:0/534083144 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26c02fb70 con 0x7ff27c152bb0 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 1 -- 10.20.193.6:0/534083144 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c037440 con 0x7ff27c152bb0 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 5 monclient: authenticate success, global_id 4641 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 4 set_mon_vals no callback set 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff27a7fc640 10 monclient: dump: 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 10 monclient: shutdown 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:08.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 1 -- 10.20.193.6:0/534083144 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c152bb0 msgr2=0x7ff27c0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 1 --2- 10.20.193.6:0/534083144 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c152bb0 0x7ff27c0c5a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0098a0 tx=0x7ff26c02ee60 comp rx=0 tx=0).stop 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 1 -- 10.20.193.6:0/534083144 shutdown_connections 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 1 --2- 10.20.193.6:0/534083144 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c152bb0 0x7ff27c0c5a80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 1 -- 10.20.193.6:0/534083144 >> 10.20.193.6:0/534083144 conn(0x7ff27c0cd780 msgr2=0x7ff27c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 1 -- 10.20.193.6:0/534083144 shutdown_connections 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.910+0000 7ff282032640 1 -- 10.20.193.6:0/534083144 wait complete. 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient: build_initial_monmap 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient: monmap: 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 librados: starting msgr at 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 librados: starting objecter 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 Processor -- start 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 -- start start 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 librados: setting wanted keys 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 librados: calling monclient init 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient: init 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c0c5fc0 0x7ff27c1f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): picked mon.noname-a con 0x7ff27c0c5fc0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff27c1527d0 0x7ff27c145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): picked mon.noname-c con 0x7ff27c1527d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff27c152bb0 0x7ff27c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): picked mon.noname-b con 0x7ff27c152bb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c06ee80 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c0c6470 con 0x7ff27c152bb0 2026-02-22T23:24:08.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c0c9570 con 0x7ff27c1527d0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.911+0000 7ff282032640 10 monclient(hunting): authenticate will time out at 1123.638988s 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c0c5fc0 0x7ff27c1f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c0c5fc0 0x7ff27c1f4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:55186/0 (socket says 10.20.193.6:55186) 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 1 -- 10.20.193.6:0/1695218837 learned_addr learned my addr 10.20.193.6:0/1695218837 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 10 monclient(hunting): get_auth_request con 0x7ff27c0c5fc0 auth_method 0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27affd640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff27c1527d0 0x7ff27c145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27bfff640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff27c152bb0 0x7ff27c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27bfff640 10 monclient(hunting): get_auth_request con 0x7ff27c152bb0 auth_method 0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27affd640 10 monclient(hunting): get_auth_request con 0x7ff27c1527d0 auth_method 0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.912+0000 7ff27affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 10 monclient(hunting): handle_auth_done global_id 4647 payload 824 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 1 -- 10.20.193.6:0/1695218837 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff27c1527d0 msgr2=0x7ff27c145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff27c1527d0 0x7ff27c145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 1 -- 10.20.193.6:0/1695218837 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff27c152bb0 msgr2=0x7ff27c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff27c152bb0 0x7ff27c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:08.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.913+0000 7ff27b7fe640 1 -- 10.20.193.6:0/1695218837 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c072da0 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.914+0000 7ff27b7fe640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c0c5fc0 0x7ff27c1f4ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c760 tx=0x7ff26800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.914+0000 7ff257fff640 10 client.?.objecter ms_handle_connect 0x7ff27c0c5fc0 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.914+0000 7ff257fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 1 -- 10.20.193.6:0/1695218837 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268017070 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: dump: 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:08.919 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: _finish_auth 0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 1 -- 10.20.193.6:0/1695218837 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff268005b90 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 5 monclient: authenticate success, global_id 4647 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 10 monclient: _renew_subs 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c073080 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 1 -- 10.20.193.6:0/1695218837 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268007070 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff279ffb640 4 set_mon_vals no callback set 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff257fff640 10 monclient: dump: 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 10 client.4647.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 10 monclient: _renew_subs 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c073640 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 1 librados: init done 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.916+0000 7ff282032640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.917+0000 7ff282032640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.917+0000 7ff282032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:08.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.917+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27c0c5a80 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.929+0000 7ff257fff640 1 -- 10.20.193.6:0/1695218837 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff268007210 con 0x7ff27c0c5fc0 2026-02-22T23:24:08.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.929+0000 7ff257fff640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff24c078e80 0x7ff24c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:08.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.929+0000 7ff27affd640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff24c078e80 0x7ff24c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:08.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.929+0000 7ff27affd640 10 monclient: get_auth_request con 0x7ff24c078e80 auth_method 0 2026-02-22T23:24:08.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.930+0000 7ff27affd640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff24c078e80 0x7ff24c07b340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff26c039000 tx=0x7ff26c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:08.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:08.930+0000 7ff257fff640 10 client.4647.objecter ms_handle_connect 0x7ff24c078e80 2026-02-22T23:24:09.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 1 -- 10.20.193.6:0/1695218837 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7ff26805d4b0 con 0x7ff27c0c5fc0 2026-02-22T23:24:09.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 10 client.4647.objecter ms_dispatch2 0x7ff27c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:09.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 3 client.4647.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:09.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 3 client.4647.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:09.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 20 client.4647.objecter dump_active .. 0 homeless 2026-02-22T23:24:09.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 1 -- 10.20.193.6:0/1695218837 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff268021020 con 0x7ff27c0c5fc0 2026-02-22T23:24:09.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:09.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.030+0000 7ff257fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:09.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.129+0000 7ff282032640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:09.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.129+0000 7ff282032640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:09.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.129+0000 7ff282032640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:09.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.129+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff27c150f10 con 0x7ff27c0c5fc0 2026-02-22T23:24:09.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.132+0000 7ff257fff640 1 -- 10.20.193.6:0/1695218837 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff268061470 con 0x7ff27c0c5fc0 2026-02-22T23:24:09.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.132+0000 7ff257fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:09.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.132+0000 7ff257fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:09.134 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:09.134 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:09.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 10 librados: watch_flush enter 2026-02-22T23:24:09.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 10 librados: watch_flush exit 2026-02-22T23:24:09.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 20 client.4647.objecter shutdown clearing up homeless session... 2026-02-22T23:24:09.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 10 client.4647.objecter successfully canceled tick 2026-02-22T23:24:09.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff24c078e80 msgr2=0x7ff24c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:09.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff24c078e80 0x7ff24c07b340 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff26c039000 tx=0x7ff26c033000 comp rx=0 tx=0).stop 2026-02-22T23:24:09.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 10 monclient: shutdown 2026-02-22T23:24:09.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:09.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c0c5fc0 msgr2=0x7ff27c1f4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:09.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c0c5fc0 0x7ff27c1f4ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c760 tx=0x7ff26800cc30 comp rx=0 tx=0).stop 2026-02-22T23:24:09.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 shutdown_connections 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff24c078e80 0x7ff24c07b340 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff27c152bb0 0x7ff27c0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff27c1527d0 0x7ff27c145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 --2- 10.20.193.6:0/1695218837 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff27c0c5fc0 0x7ff27c1f4ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 >> 10.20.193.6:0/1695218837 conn(0x7ff27c0cd780 msgr2=0x7ff27c0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 shutdown_connections 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 -- 10.20.193.6:0/1695218837 wait complete. 2026-02-22T23:24:09.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:09.133+0000 7ff282032640 1 librados: shutdown 2026-02-22T23:24:12.144 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:12.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.211+0000 7f16cecc9640 10 monclient: get_monmap_and_config 2026-02-22T23:24:12.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.211+0000 7f16cecc9640 10 monclient: build_initial_monmap 2026-02-22T23:24:12.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.211+0000 7f16cecc9640 10 monclient: monmap: 2026-02-22T23:24:12.217 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:12.217 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:12.217 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:12.211889+0000 2026-02-22T23:24:12.217 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:12.211889+0000 2026-02-22T23:24:12.217 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:12.217 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:12.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.211+0000 7f16cecc9640 1 Processor -- start 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.211+0000 7f16cecc9640 1 -- start start 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.211+0000 7f16cecc9640 10 monclient: init 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient: _add_conns ranks=[5,3,0,2,4,1] 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f16c81527d0 0x7f16c806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient(hunting): picked mon.noname-c con 0x7f16c81527d0 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f16c8152bb0 0x7f16c80c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:12.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient(hunting): picked mon.noname-a con 0x7f16c8152bb0 addr v2:10.20.193.6:3300/0 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f16c80c5fc0 addr v1:10.20.193.6:6789/0 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c806e300 con 0x7f16c80c5fc0 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f16c806f1f0 con 0x7f16c8152bb0 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f16c80c6470 con 0x7f16c81527d0 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:12.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cecc9640 10 monclient(hunting): authenticate will time out at 1126.939360s 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16bffff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f16c8152bb0 0x7f16c80c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.212+0000 7f16cca3e640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f16c81527d0 0x7f16c806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cd23f640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f16c80c5fc0 0x7f16c80c8e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:53630/0 (socket says 10.20.193.6:53630) 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cd23f640 1 -- 10.20.193.6:0/2268777619 learned_addr learned my addr 10.20.193.6:0/2268777619 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16bffff640 10 monclient(hunting): get_auth_request con 0x7f16c8152bb0 auth_method 0 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16bffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:12.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cca3e640 10 monclient(hunting): get_auth_request con 0x7f16c81527d0 auth_method 0 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cca3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cca3e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cca3e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cca3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cca3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16cca3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:12.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.213+0000 7f16bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 10 monclient(hunting): handle_auth_done global_id 4650 payload 274 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 1 -- 10.20.193.6:0/2268777619 >> v2:10.20.193.80:3300/0 conn(0x7f16c81527d0 msgr2=0x7f16c806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 1 --2- 10.20.193.6:0/2268777619 >> v2:10.20.193.80:3300/0 conn(0x7f16c81527d0 0x7f16c806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 1 -- 10.20.193.6:0/2268777619 >> v1:10.20.193.6:6789/0 conn(0x7f16c80c5fc0 legacy=0x7f16c80c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 1 monclient: found mon.noname-a 2026-02-22T23:24:12.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:12.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:12.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.214+0000 7f16bffff640 1 -- 10.20.193.6:0/2268777619 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c806e5e0 con 0x7f16c8152bb0 2026-02-22T23:24:12.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.216+0000 7f16bffff640 1 --2- 10.20.193.6:0/2268777619 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c8152bb0 0x7f16c80c5a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16c80caec0 tx=0x7f16b002ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c285fe264bcc4101 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:12.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 1 -- 10.20.193.6:0/2268777619 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b003c070 con 0x7f16c8152bb0 2026-02-22T23:24:12.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: dump: 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:12.234 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: _finish_auth 0 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 1 -- 10.20.193.6:0/2268777619 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b002faf0 con 0x7f16c8152bb0 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 1 -- 10.20.193.6:0/2268777619 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0037440 con 0x7f16c8152bb0 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16bf7fe640 10 monclient: dump: 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:12.235 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 5 monclient: authenticate success, global_id 4650 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 4 set_mon_vals no callback set 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 10 monclient: shutdown 2026-02-22T23:24:12.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 -- 10.20.193.6:0/2268777619 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c8152bb0 msgr2=0x7f16c80c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2268777619 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c8152bb0 0x7f16c80c5a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16c80caec0 tx=0x7f16b002ede0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 -- 10.20.193.6:0/2268777619 shutdown_connections 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2268777619 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c8152bb0 0x7f16c80c5a80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2268777619 >> v2:10.20.193.80:3300/0 conn(0x7f16c81527d0 0x7f16c806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 -- 10.20.193.6:0/2268777619 >> 10.20.193.6:0/2268777619 conn(0x7f16c80cd780 msgr2=0x7f16c80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 -- 10.20.193.6:0/2268777619 shutdown_connections 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.218+0000 7f16cecc9640 1 -- 10.20.193.6:0/2268777619 wait complete. 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 10 monclient: build_initial_monmap 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 10 monclient: monmap: 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:12.237 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 1 librados: starting msgr at 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 1 librados: starting objecter 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 1 Processor -- start 2026-02-22T23:24:12.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 1 -- start start 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 1 librados: setting wanted keys 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 1 librados: calling monclient init 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.219+0000 7f16cecc9640 10 monclient: init 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16c80c5fc0 0x7f16c81f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): picked mon.noname-b con 0x7f16c80c5fc0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c81527d0 0x7f16c8147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): picked mon.noname-a con 0x7f16c81527d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16c8152bb0 0x7f16c80bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): picked mon.noname-c con 0x7f16c8152bb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f16c806ee80 con 0x7f16c81527d0 2026-02-22T23:24:12.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f16c806eba0 con 0x7f16c80c5fc0 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f16c806dfa0 con 0x7f16c8152bb0 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cecc9640 10 monclient(hunting): authenticate will time out at 1126.947086s 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c81527d0 0x7f16c8147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c81527d0 0x7f16c8147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:55198/0 (socket says 10.20.193.6:55198) 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 1 -- 10.20.193.6:0/2753427718 learned_addr learned my addr 10.20.193.6:0/2753427718 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 10 monclient(hunting): get_auth_request con 0x7f16c81527d0 auth_method 0 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:12.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16c8152bb0 0x7f16c80bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16bffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cca3e640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16c80c5fc0 0x7f16c81f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 10 monclient(hunting): get_auth_request con 0x7f16c8152bb0 auth_method 0 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cca3e640 10 monclient(hunting): get_auth_request con 0x7f16c80c5fc0 auth_method 0 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cca3e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cca3e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:12.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cca3e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.220+0000 7f16cd23f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.221+0000 7f16cca3e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.221+0000 7f16cca3e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.221+0000 7f16cca3e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 10 monclient(hunting): handle_auth_done global_id 4656 payload 824 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 1 -- 10.20.193.6:0/2753427718 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16c8152bb0 msgr2=0x7f16c80bfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16c8152bb0 0x7f16c80bfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 1 -- 10.20.193.6:0/2753427718 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16c80c5fc0 msgr2=0x7f16c81f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16c80c5fc0 0x7f16c81f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 1 monclient: found mon.noname-a 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.222+0000 7f16bffff640 1 -- 10.20.193.6:0/2753427718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c80c0640 con 0x7f16c81527d0 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.223+0000 7f16bffff640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c81527d0 0x7f16c8147380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009130 tx=0x7f16b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.223+0000 7f16bd7fa640 10 client.?.objecter ms_handle_connect 0x7f16c81527d0 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.223+0000 7f16bd7fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 1 -- 10.20.193.6:0/2753427718 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b003c040 con 0x7f16c81527d0 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: dump: 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:12.243 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: _finish_auth 0 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: _check_auth_tickets 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 1 -- 10.20.193.6:0/2753427718 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b0042610 con 0x7f16c81527d0 2026-02-22T23:24:12.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 1 -- 10.20.193.6:0/2753427718 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b00429e0 con 0x7f16c81527d0 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16beffd640 4 set_mon_vals no callback set 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16bd7fa640 10 monclient: dump: 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:12.245 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 5 monclient: authenticate success, global_id 4656 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 10 monclient: _renew_subs 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c80c0920 con 0x7f16c81527d0 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 10 client.4656.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 10 monclient: _renew_subs 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c80bb6d0 con 0x7f16c81527d0 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.224+0000 7f16cecc9640 1 librados: init done 2026-02-22T23:24:12.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.225+0000 7f16cecc9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:12.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.225+0000 7f16cecc9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:12.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.225+0000 7f16cecc9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:12.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.225+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c806da60 con 0x7f16c81527d0 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.236+0000 7f16bd7fa640 1 -- 10.20.193.6:0/2753427718 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f16b004b450 con 0x7f16c81527d0 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.236+0000 7f16bd7fa640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1698078e50 0x7f169807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.236+0000 7f16cca3e640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1698078e50 0x7f169807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.236+0000 7f16cca3e640 10 monclient: get_auth_request con 0x7f1698078e50 auth_method 0 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.237+0000 7f16cca3e640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1698078e50 0x7f169807b310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009c40 tx=0x7f16b8009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.237+0000 7f16bd7fa640 10 client.4656.objecter ms_handle_connect 0x7f1698078e50 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.238+0000 7f16bd7fa640 1 -- 10.20.193.6:0/2753427718 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f16b002fa70 con 0x7f16c81527d0 2026-02-22T23:24:12.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.238+0000 7f16bd7fa640 10 client.4656.objecter ms_dispatch2 0x7f16c8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:12.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.238+0000 7f16bd7fa640 3 client.4656.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:12.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.238+0000 7f16bd7fa640 3 client.4656.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:12.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.238+0000 7f16bd7fa640 20 client.4656.objecter dump_active .. 0 homeless 2026-02-22T23:24:12.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.335+0000 7f16bd7fa640 1 -- 10.20.193.6:0/2753427718 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f16b0086820 con 0x7f16c81527d0 2026-02-22T23:24:12.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.335+0000 7f16bd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:12.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.335+0000 7f16bd7fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:12.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.436+0000 7f16cecc9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:12.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.436+0000 7f16cecc9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:12.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.436+0000 7f16cecc9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:12.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.436+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f16c80ca500 con 0x7f16c81527d0 2026-02-22T23:24:12.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.439+0000 7f16bd7fa640 1 -- 10.20.193.6:0/2753427718 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f16b008ab20 con 0x7f16c81527d0 2026-02-22T23:24:12.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.439+0000 7f16bd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:12.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.439+0000 7f16bd7fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:12.441 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:12.442 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.440+0000 7f16cecc9640 10 librados: watch_flush enter 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 10 librados: watch_flush exit 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 20 client.4656.objecter shutdown clearing up homeless session... 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 10 client.4656.objecter successfully canceled tick 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1698078e50 msgr2=0x7f169807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1698078e50 0x7f169807b310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009c40 tx=0x7f16b8009290 comp rx=0 tx=0).stop 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 10 monclient: shutdown 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:12.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c81527d0 msgr2=0x7f16c8147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c81527d0 0x7f16c8147380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009130 tx=0x7f16b00047c0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 shutdown_connections 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1698078e50 0x7f169807b310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f16c8152bb0 0x7f16c80bfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f16c81527d0 0x7f16c8147380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 --2- 10.20.193.6:0/2753427718 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f16c80c5fc0 0x7f16c81f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 >> 10.20.193.6:0/2753427718 conn(0x7f16c80cd780 msgr2=0x7f16c8150f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 shutdown_connections 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 -- 10.20.193.6:0/2753427718 wait complete. 2026-02-22T23:24:12.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:12.441+0000 7f16cecc9640 1 librados: shutdown 2026-02-22T23:24:12.932 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:57.182 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.960 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:57.209 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.960 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:57.206 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.962 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:57.208 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.968 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:02:57.218 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.969 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:02:57.216 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.986 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:57.197 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.988 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:57.199 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:12.996 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:02:57.207 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:15.452 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:15.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.520+0000 7fbf020bd640 10 monclient: get_monmap_and_config 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.520+0000 7fbf020bd640 10 monclient: build_initial_monmap 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.520+0000 7fbf020bd640 10 monclient: monmap: 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:15.521163+0000 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:15.521163+0000 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:15.522 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.520+0000 7fbf020bd640 1 Processor -- start 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.520+0000 7fbf020bd640 1 -- start start 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.520+0000 7fbf020bd640 10 monclient: init 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient: _add_conns ranks=[5,0,4,2,1,3] 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fbefc06ff20 0x7fbefc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient(hunting): picked mon.noname-c con 0x7fbefc06ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fbefc15c970 addr v1:10.20.193.6:6789/0 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fbefc155eb0 0x7fbefc15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient(hunting): picked mon.noname-b con 0x7fbefc155eb0 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbefc0725b0 con 0x7fbefc15c970 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc072890 con 0x7fbefc155eb0 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc072b70 con 0x7fbefc06ff20 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:15.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbf020bd640 10 monclient(hunting): authenticate will time out at 1130.248527s 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbefaffd640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fbefc15c970 0x7fbefc1557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:53636/0 (socket says 10.20.193.6:53636) 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.521+0000 7fbefaffd640 1 -- 10.20.193.6:0/2257912590 learned_addr learned my addr 10.20.193.6:0/2257912590 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 1 --2- 10.20.193.6:0/2257912590 >> v2:10.20.193.40:3300/0 conn(0x7fbefc155eb0 0x7fbefc15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 1 --2- 10.20.193.6:0/2257912590 >> v2:10.20.193.80:3300/0 conn(0x7fbefc06ff20 0x7fbefc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 10 monclient(hunting): get_auth_request con 0x7fbefc155eb0 auth_method 0 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 10 monclient(hunting): get_auth_request con 0x7fbefc06ff20 auth_method 0 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:15.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.522+0000 7fbefb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefa7fc640 1 -- 10.20.193.6:0/2257912590 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229288213 0 0) 0x7fbefc0725b0 con 0x7fbefc15c970 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefa7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefa7fc640 10 monclient(hunting): my global_id is 4659 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefa7fc640 1 -- 10.20.193.6:0/2257912590 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbee0003620 con 0x7fbefc15c970 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 10 monclient(hunting): handle_auth_done global_id 4669 payload 274 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 1 -- 10.20.193.6:0/2257912590 >> v2:10.20.193.80:3300/0 conn(0x7fbefc06ff20 msgr2=0x7fbefc070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 1 --2- 10.20.193.6:0/2257912590 >> v2:10.20.193.80:3300/0 conn(0x7fbefc06ff20 0x7fbefc070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 1 -- 10.20.193.6:0/2257912590 >> v1:10.20.193.6:6789/0 conn(0x7fbefc15c970 legacy=0x7fbefc1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:15.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:15.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 1 monclient: found mon.noname-b 2026-02-22T23:24:15.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:15.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:24:15.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.523+0000 7fbefbfff640 1 -- 10.20.193.6:0/2257912590 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc072e50 con 0x7fbefc155eb0 2026-02-22T23:24:15.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.524+0000 7fbefbfff640 1 --2- 10.20.193.6:0/2257912590 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc155eb0 0x7fbefc15d1c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbefc156590 tx=0x7fbef002f1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5e6b3ca22b31e2a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 1 -- 10.20.193.6:0/2257912590 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef003c070 con 0x7fbefc155eb0 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: dump: 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: _finish_auth 0 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:15.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 1 -- 10.20.193.6:0/2257912590 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef0037440 con 0x7fbefc155eb0 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 5 monclient: authenticate success, global_id 4669 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 1 -- 10.20.193.6:0/2257912590 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0037810 con 0x7fbefc155eb0 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbefa7fc640 10 monclient: dump: 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 4 set_mon_vals no callback set 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 10 monclient: shutdown 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 -- 10.20.193.6:0/2257912590 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc155eb0 msgr2=0x7fbefc15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 --2- 10.20.193.6:0/2257912590 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc155eb0 0x7fbefc15d1c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbefc156590 tx=0x7fbef002f1c0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 -- 10.20.193.6:0/2257912590 shutdown_connections 2026-02-22T23:24:15.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 --2- 10.20.193.6:0/2257912590 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc155eb0 0x7fbefc15d1c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 --2- 10.20.193.6:0/2257912590 >> v2:10.20.193.80:3300/0 conn(0x7fbefc06ff20 0x7fbefc070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 -- 10.20.193.6:0/2257912590 >> 10.20.193.6:0/2257912590 conn(0x7fbefc0cd780 msgr2=0x7fbefc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:15.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 -- 10.20.193.6:0/2257912590 shutdown_connections 2026-02-22T23:24:15.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.526+0000 7fbf020bd640 1 -- 10.20.193.6:0/2257912590 wait complete. 2026-02-22T23:24:15.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 10 monclient: build_initial_monmap 2026-02-22T23:24:15.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 10 monclient: monmap: 2026-02-22T23:24:15.531 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 1 librados: starting msgr at 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 1 librados: starting objecter 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 1 Processor -- start 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 1 -- start start 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 1 librados: setting wanted keys 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 1 librados: calling monclient init 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.527+0000 7fbf020bd640 10 monclient: init 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc06ff20 0x7fbefc1f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): picked mon.noname-b con 0x7fbefc06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbefc155eb0 0x7fbefc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): picked mon.noname-c con 0x7fbefc155eb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbefc15c970 0x7fbefc14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:15.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): picked mon.noname-a con 0x7fbefc15c970 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc0737d0 con 0x7fbefc15c970 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc073650 con 0x7fbefc06ff20 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc073950 con 0x7fbefc155eb0 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbf020bd640 10 monclient(hunting): authenticate will time out at 1130.255082s 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbefc15c970 0x7fbefc14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbefc15c970 0x7fbefc14fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:55206/0 (socket says 10.20.193.6:55206) 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 1 -- 10.20.193.6:0/3792315910 learned_addr learned my addr 10.20.193.6:0/3792315910 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc06ff20 0x7fbefc1f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 10 monclient(hunting): get_auth_request con 0x7fbefc15c970 auth_method 0 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefaffd640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbefc155eb0 0x7fbefc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 10 monclient(hunting): get_auth_request con 0x7fbefc06ff20 auth_method 0 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefaffd640 10 monclient(hunting): get_auth_request con 0x7fbefc155eb0 auth_method 0 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:15.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefaffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.528+0000 7fbefb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 10 monclient(hunting): handle_auth_done global_id 4662 payload 824 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 1 -- 10.20.193.6:0/3792315910 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbefc155eb0 msgr2=0x7fbefc15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbefc155eb0 0x7fbefc15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 1 -- 10.20.193.6:0/3792315910 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc06ff20 msgr2=0x7fbefc1f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc06ff20 0x7fbefc1f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 1 monclient: found mon.noname-a 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:15.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.529+0000 7fbefbfff640 1 -- 10.20.193.6:0/3792315910 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc1e7270 con 0x7fbefc15c970 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.530+0000 7fbefbfff640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbefc15c970 0x7fbefc14fcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbef002f6f0 tx=0x7fbef0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.530+0000 7fbed7fff640 10 client.?.objecter ms_handle_connect 0x7fbefc15c970 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.530+0000 7fbed7fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 1 -- 10.20.193.6:0/3792315910 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef003c050 con 0x7fbefc15c970 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: dump: 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: _finish_auth 0 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:15.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 1 -- 10.20.193.6:0/3792315910 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef0048070 con 0x7fbefc15c970 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 1 -- 10.20.193.6:0/3792315910 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0041a50 con 0x7fbefc15c970 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbed7fff640 10 monclient: dump: 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbef9ffb640 4 set_mon_vals no callback set 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 5 monclient: authenticate success, global_id 4662 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 10 monclient: _renew_subs 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc15b730 con 0x7fbefc15c970 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 10 client.4662.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 10 monclient: _renew_subs 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc1f3e30 con 0x7fbefc15c970 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.532+0000 7fbf020bd640 1 librados: init done 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.533+0000 7fbf020bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.533+0000 7fbf020bd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.533+0000 7fbf020bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:15.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.533+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc070300 con 0x7fbefc15c970 2026-02-22T23:24:15.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.636+0000 7fbed7fff640 1 -- 10.20.193.6:0/3792315910 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fbef0041bf0 con 0x7fbefc15c970 2026-02-22T23:24:15.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.636+0000 7fbed7fff640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbecc078e00 0x7fbecc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:15.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.637+0000 7fbefb7fe640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbecc078e00 0x7fbecc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:15.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.637+0000 7fbefb7fe640 10 monclient: get_auth_request con 0x7fbecc078e00 auth_method 0 2026-02-22T23:24:15.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.637+0000 7fbed7fff640 10 client.4662.objecter ms_handle_connect 0x7fbecc078e00 2026-02-22T23:24:15.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.637+0000 7fbefb7fe640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbecc078e00 0x7fbecc07b2c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009ba0 tx=0x7fbee8009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 1 -- 10.20.193.6:0/3792315910 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fbef00bdb80 con 0x7fbefc15c970 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 10 client.4662.objecter ms_dispatch2 0x7fbefc000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 3 client.4662.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 3 client.4662.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 20 client.4662.objecter dump_active .. 0 homeless 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 1 -- 10.20.193.6:0/3792315910 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fbef0088350 con 0x7fbefc15c970 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:15.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.644+0000 7fbed7fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:15.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.744+0000 7fbf020bd640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:15.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.744+0000 7fbf020bd640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:15.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.744+0000 7fbf020bd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:15.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.744+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbefc15c750 con 0x7fbefc15c970 2026-02-22T23:24:15.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.746+0000 7fbed7fff640 1 -- 10.20.193.6:0/3792315910 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fbef008c650 con 0x7fbefc15c970 2026-02-22T23:24:15.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.747+0000 7fbed7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:15.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.747+0000 7fbed7fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:15.748 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:15.748 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 10 librados: watch_flush enter 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 10 librados: watch_flush exit 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 20 client.4662.objecter shutdown clearing up homeless session... 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 10 client.4662.objecter successfully canceled tick 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbecc078e00 msgr2=0x7fbecc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbecc078e00 0x7fbecc07b2c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009ba0 tx=0x7fbee8009290 comp rx=0 tx=0).stop 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 10 monclient: shutdown 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbefc15c970 msgr2=0x7fbefc14fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbefc15c970 0x7fbefc14fcd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbef002f6f0 tx=0x7fbef0004b40 comp rx=0 tx=0).stop 2026-02-22T23:24:15.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 shutdown_connections 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fbecc078e00 0x7fbecc07b2c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fbefc15c970 0x7fbefc14fcd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fbefc155eb0 0x7fbefc15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 --2- 10.20.193.6:0/3792315910 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fbefc06ff20 0x7fbefc1f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 >> 10.20.193.6:0/3792315910 conn(0x7fbefc0cd780 msgr2=0x7fbefc155570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 shutdown_connections 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 -- 10.20.193.6:0/3792315910 wait complete. 2026-02-22T23:24:15.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:15.748+0000 7fbf020bd640 1 librados: shutdown 2026-02-22T23:24:18.758 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:18.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.824+0000 7fae23986640 10 monclient: get_monmap_and_config 2026-02-22T23:24:18.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.824+0000 7fae23986640 10 monclient: build_initial_monmap 2026-02-22T23:24:18.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.824+0000 7fae23986640 10 monclient: monmap: 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:18.825151+0000 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:18.825151+0000 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:18.827 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:18.828 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:18.828 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:18.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.824+0000 7fae23986640 1 Processor -- start 2026-02-22T23:24:18.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.824+0000 7fae23986640 1 -- start start 2026-02-22T23:24:18.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.824+0000 7fae23986640 10 monclient: init 2026-02-22T23:24:18.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient: _add_conns ranks=[4,1,3,0,2,5] 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fae1c06ff20 0x7fae1c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient(hunting): picked mon.noname-b con 0x7fae1c06ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fae1c0c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fae1c15a5e0 0x7fae1c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient(hunting): picked mon.noname-a con 0x7fae1c15a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae1c0c7e90 con 0x7fae1c0c99a0 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fae1c159a40 con 0x7fae1c15a5e0 2026-02-22T23:24:18.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fae1c0c8170 con 0x7fae1c06ff20 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae23986640 10 monclient(hunting): authenticate will time out at 1133.552520s 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.825+0000 7fae21efc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fae1c15a5e0 0x7fae1c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fae1c06ff20 0x7fae1c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fae1c15a5e0 0x7fae1c14fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:50660/0 (socket says 10.20.193.6:50660) 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 1 -- 10.20.193.6:0/3970129863 learned_addr learned my addr 10.20.193.6:0/3970129863 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 10 monclient(hunting): get_auth_request con 0x7fae1c15a5e0 auth_method 0 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:18.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 10 monclient(hunting): get_auth_request con 0x7fae1c06ff20 auth_method 0 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae21efc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.826+0000 7fae216fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 10 monclient(hunting): handle_auth_done global_id 4665 payload 274 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 1 -- 10.20.193.6:0/3970129863 >> v2:10.20.193.40:3300/0 conn(0x7fae1c06ff20 msgr2=0x7fae1c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 1 --2- 10.20.193.6:0/3970129863 >> v2:10.20.193.40:3300/0 conn(0x7fae1c06ff20 0x7fae1c070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:18.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 1 -- 10.20.193.6:0/3970129863 >> v1:10.20.193.40:6789/0 conn(0x7fae1c0c99a0 legacy=0x7fae1c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:18.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 10 monclient: _finish_hunting 0 2026-02-22T23:24:18.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 1 monclient: found mon.noname-a 2026-02-22T23:24:18.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:18.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:18.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.827+0000 7fae21efc640 1 -- 10.20.193.6:0/3970129863 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c0c8450 con 0x7fae1c15a5e0 2026-02-22T23:24:18.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.828+0000 7fae21efc640 1 --2- 10.20.193.6:0/3970129863 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c15a5e0 0x7fae1c14fe10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae18009e50 tx=0x7fae1802f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a5ccc33bb8a393 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 1 -- 10.20.193.6:0/3970129863 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1802fcc0 con 0x7fae1c15a5e0 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: dump: 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: _finish_auth 0 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 1 -- 10.20.193.6:0/3970129863 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae180373f0 con 0x7fae1c15a5e0 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 5 monclient: authenticate success, global_id 4665 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 1 -- 10.20.193.6:0/3970129863 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1802fe60 con 0x7fae1c15a5e0 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae0bfff640 10 monclient: dump: 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:18.839 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 4 set_mon_vals no callback set 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 10 monclient: shutdown 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 -- 10.20.193.6:0/3970129863 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c15a5e0 msgr2=0x7fae1c14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 --2- 10.20.193.6:0/3970129863 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c15a5e0 0x7fae1c14fe10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae18009e50 tx=0x7fae1802f2a0 comp rx=0 tx=0).stop 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 -- 10.20.193.6:0/3970129863 shutdown_connections 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 --2- 10.20.193.6:0/3970129863 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c15a5e0 0x7fae1c14fe10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 --2- 10.20.193.6:0/3970129863 >> v2:10.20.193.40:3300/0 conn(0x7fae1c06ff20 0x7fae1c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 -- 10.20.193.6:0/3970129863 >> 10.20.193.6:0/3970129863 conn(0x7fae1c0cd780 msgr2=0x7fae1c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 -- 10.20.193.6:0/3970129863 shutdown_connections 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 1 -- 10.20.193.6:0/3970129863 wait complete. 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 10 monclient: build_initial_monmap 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.830+0000 7fae23986640 10 monclient: monmap: 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 librados: starting msgr at 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 librados: starting objecter 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 Processor -- start 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 -- start start 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 librados: setting wanted keys 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 librados: calling monclient init 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient: init 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fae1c06ff20 0x7fae1c0c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): picked mon.noname-c con 0x7fae1c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c0c99a0 0x7fae1c1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): picked mon.noname-a con 0x7fae1c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fae1c15a5e0 0x7fae1c0c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): picked mon.noname-b con 0x7fae1c15a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:18.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c150350 con 0x7fae1c0c99a0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c0c8a10 con 0x7fae1c15a5e0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c1504d0 con 0x7fae1c06ff20 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae23986640 10 monclient(hunting): authenticate will time out at 1133.558654s 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.831+0000 7fae20efa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c0c99a0 0x7fae1c1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c0c99a0 0x7fae1c1f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:50672/0 (socket says 10.20.193.6:50672) 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 1 -- 10.20.193.6:0/4244672344 learned_addr learned my addr 10.20.193.6:0/4244672344 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 10 monclient(hunting): get_auth_request con 0x7fae1c0c99a0 auth_method 0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fae1c06ff20 0x7fae1c0c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fae1c15a5e0 0x7fae1c0c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae20efa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 10 monclient(hunting): get_auth_request con 0x7fae1c06ff20 auth_method 0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 10 monclient(hunting): get_auth_request con 0x7fae1c15a5e0 auth_method 0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae216fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.832+0000 7fae21efc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 10 monclient(hunting): handle_auth_done global_id 4668 payload 824 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 1 -- 10.20.193.6:0/4244672344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fae1c06ff20 msgr2=0x7fae1c0c09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fae1c06ff20 0x7fae1c0c09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 1 -- 10.20.193.6:0/4244672344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fae1c15a5e0 msgr2=0x7fae1c0c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fae1c15a5e0 0x7fae1c0c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 10 monclient: _finish_hunting 0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 1 monclient: found mon.noname-a 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.833+0000 7fae20efa640 1 -- 10.20.193.6:0/4244672344 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c1e6dd0 con 0x7fae1c0c99a0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.834+0000 7fae20efa640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c0c99a0 0x7fae1c1f26b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae1000b570 tx=0x7fae1000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:18.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.834+0000 7fae09ffb640 10 client.?.objecter ms_handle_connect 0x7fae1c0c99a0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.834+0000 7fae09ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 1 -- 10.20.193.6:0/4244672344 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae10013020 con 0x7fae1c0c99a0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: dump: 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: _finish_auth 0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 1 -- 10.20.193.6:0/4244672344 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae100044a0 con 0x7fae1c0c99a0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 1 -- 10.20.193.6:0/4244672344 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae100029c0 con 0x7fae1c0c99a0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 5 monclient: authenticate success, global_id 4668 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 monclient: _renew_subs 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c0bfbe0 con 0x7fae1c0c99a0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae0b7fe640 4 set_mon_vals no callback set 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae09ffb640 10 monclient: dump: 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 client.4668.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 monclient: _renew_subs 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c0c01a0 con 0x7fae1c0c99a0 2026-02-22T23:24:18.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 1 librados: init done 2026-02-22T23:24:18.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:18.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:18.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:18.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.836+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1c0c04d0 con 0x7fae1c0c99a0 2026-02-22T23:24:18.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.848+0000 7fae09ffb640 1 -- 10.20.193.6:0/4244672344 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fae10002b60 con 0x7fae1c0c99a0 2026-02-22T23:24:18.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.848+0000 7fae09ffb640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fadec078e80 0x7fadec07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:18.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.848+0000 7fae216fb640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fadec078e80 0x7fadec07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:18.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.848+0000 7fae216fb640 10 monclient: get_auth_request con 0x7fadec078e80 auth_method 0 2026-02-22T23:24:18.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.849+0000 7fae216fb640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fadec078e80 0x7fadec07b340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009c40 tx=0x7fae0c009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:18.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.849+0000 7fae09ffb640 10 client.4668.objecter ms_handle_connect 0x7fadec078e80 2026-02-22T23:24:18.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.868+0000 7fae09ffb640 1 -- 10.20.193.6:0/4244672344 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fae1005dff0 con 0x7fae1c0c99a0 2026-02-22T23:24:18.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.868+0000 7fae09ffb640 10 client.4668.objecter ms_dispatch2 0x7fae1c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:18.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.868+0000 7fae09ffb640 3 client.4668.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:18.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.868+0000 7fae09ffb640 3 client.4668.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:18.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.868+0000 7fae09ffb640 20 client.4668.objecter dump_active .. 0 homeless 2026-02-22T23:24:18.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.944+0000 7fae09ffb640 1 -- 10.20.193.6:0/4244672344 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fae1001d020 con 0x7fae1c0c99a0 2026-02-22T23:24:18.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.944+0000 7fae09ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:18.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:18.944+0000 7fae09ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:19.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.045+0000 7fae23986640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:19.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.045+0000 7fae23986640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:19.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.045+0000 7fae23986640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:19.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.045+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fae1c150860 con 0x7fae1c0c99a0 2026-02-22T23:24:19.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.048+0000 7fae09ffb640 1 -- 10.20.193.6:0/4244672344 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fae10061fb0 con 0x7fae1c0c99a0 2026-02-22T23:24:19.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.048+0000 7fae09ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:19.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.048+0000 7fae09ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:19.050 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:19.051 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:19.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.049+0000 7fae23986640 10 librados: watch_flush enter 2026-02-22T23:24:19.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.049+0000 7fae23986640 10 librados: watch_flush exit 2026-02-22T23:24:19.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.049+0000 7fae23986640 20 client.4668.objecter shutdown clearing up homeless session... 2026-02-22T23:24:19.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.049+0000 7fae23986640 10 client.4668.objecter successfully canceled tick 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fadec078e80 msgr2=0x7fadec07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fadec078e80 0x7fadec07b340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae0c009c40 tx=0x7fae0c009290 comp rx=0 tx=0).stop 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 10 monclient: shutdown 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c0c99a0 msgr2=0x7fae1c1f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c0c99a0 0x7fae1c1f26b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae1000b570 tx=0x7fae1000ba40 comp rx=0 tx=0).stop 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 shutdown_connections 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fadec078e80 0x7fadec07b340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fae1c15a5e0 0x7fae1c0c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fae1c0c99a0 0x7fae1c1f26b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 --2- 10.20.193.6:0/4244672344 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fae1c06ff20 0x7fae1c0c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:19.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 >> 10.20.193.6:0/4244672344 conn(0x7fae1c0cd780 msgr2=0x7fae1c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:19.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 shutdown_connections 2026-02-22T23:24:19.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 -- 10.20.193.6:0/4244672344 wait complete. 2026-02-22T23:24:19.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:19.050+0000 7fae23986640 1 librados: shutdown 2026-02-22T23:24:22.061 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.128+0000 7f680a2bf640 10 monclient: get_monmap_and_config 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.128+0000 7f680a2bf640 10 monclient: build_initial_monmap 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.128+0000 7f680a2bf640 10 monclient: monmap: 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:22.128817+0000 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:22.128817+0000 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:22.130 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.128+0000 7f680a2bf640 1 Processor -- start 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.128+0000 7f680a2bf640 1 -- start start 2026-02-22T23:24:22.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.128+0000 7f680a2bf640 10 monclient: init 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient: _add_conns ranks=[5,3,1,2,4,0] 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f680406ff20 0x7f6804070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient(hunting): picked mon.noname-c con 0x7f680406ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f68040c99a0 0x7f6804159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient(hunting): picked mon.noname-a con 0x7f68040c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f680415a410 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f680415a860 con 0x7f680415a410 2026-02-22T23:24:22.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6804159a40 con 0x7f68040c99a0 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f68040c7e90 con 0x7f680406ff20 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f680a2bf640 10 monclient(hunting): authenticate will time out at 1136.856182s 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f680406ff20 0x7f6804070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f68037fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f68040c99a0 0x7f6804159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6808835640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f680415a410 0x7f680414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:57610/0 (socket says 10.20.193.6:57610) 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6808835640 1 -- 10.20.193.6:0/2322112365 learned_addr learned my addr 10.20.193.6:0/2322112365 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 10 monclient(hunting): get_auth_request con 0x7f680406ff20 auth_method 0 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:22.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f68037fe640 10 monclient(hunting): get_auth_request con 0x7f68040c99a0 auth_method 0 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f68037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f68037fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f68037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.129+0000 7f6803fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f68037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f68037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f68037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 10 monclient(hunting): handle_auth_done global_id 4760 payload 274 2026-02-22T23:24:22.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 1 -- 10.20.193.6:0/2322112365 >> v2:10.20.193.6:3300/0 conn(0x7f68040c99a0 msgr2=0x7f6804159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 1 --2- 10.20.193.6:0/2322112365 >> v2:10.20.193.6:3300/0 conn(0x7f68040c99a0 0x7f6804159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 1 -- 10.20.193.6:0/2322112365 >> v1:10.20.193.40:6789/0 conn(0x7f680415a410 legacy=0x7f680414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 1 monclient: found mon.noname-c 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.130+0000 7f6803fff640 1 -- 10.20.193.6:0/2322112365 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68040c8170 con 0x7f680406ff20 2026-02-22T23:24:22.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.131+0000 7f6803fff640 1 --2- 10.20.193.6:0/2322112365 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680406ff20 0x7f6804070300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67f00098a0 tx=0x7f67f002ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa901d202c93e00c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:22.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 1 -- 10.20.193.6:0/2322112365 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f003c070 con 0x7f680406ff20 2026-02-22T23:24:22.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: dump: 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:22.144 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: _finish_auth 0 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 5 monclient: authenticate success, global_id 4760 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 1 -- 10.20.193.6:0/2322112365 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f002fae0 con 0x7f680406ff20 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 1 -- 10.20.193.6:0/2322112365 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0037440 con 0x7f680406ff20 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:22.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f6802ffd640 10 monclient: dump: 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 4 set_mon_vals no callback set 2026-02-22T23:24:22.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 10 monclient: shutdown 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 1 -- 10.20.193.6:0/2322112365 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680406ff20 msgr2=0x7f6804070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 1 --2- 10.20.193.6:0/2322112365 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680406ff20 0x7f6804070300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f67f00098a0 tx=0x7f67f002ee10 comp rx=0 tx=0).stop 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 1 -- 10.20.193.6:0/2322112365 shutdown_connections 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 1 --2- 10.20.193.6:0/2322112365 >> v2:10.20.193.6:3300/0 conn(0x7f68040c99a0 0x7f6804159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.133+0000 7f680a2bf640 1 --2- 10.20.193.6:0/2322112365 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680406ff20 0x7f6804070300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.134+0000 7f680a2bf640 1 -- 10.20.193.6:0/2322112365 >> 10.20.193.6:0/2322112365 conn(0x7f68040cd780 msgr2=0x7f68040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.134+0000 7f680a2bf640 1 -- 10.20.193.6:0/2322112365 shutdown_connections 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.134+0000 7f680a2bf640 1 -- 10.20.193.6:0/2322112365 wait complete. 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.134+0000 7f680a2bf640 10 monclient: build_initial_monmap 2026-02-22T23:24:22.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.134+0000 7f680a2bf640 10 monclient: monmap: 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 librados: starting msgr at 2026-02-22T23:24:22.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 librados: starting objecter 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 Processor -- start 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 -- start start 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 librados: setting wanted keys 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 librados: calling monclient init 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient: init 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f680406ff20 0x7f6804156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): picked mon.noname-b con 0x7f680406ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f68040c99a0 0x7f68041f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): picked mon.noname-a con 0x7f68040c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680415a410 0x7f680406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:22.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): picked mon.noname-c con 0x7f680415a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f68040c8a10 con 0x7f68040c99a0 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f68040c8730 con 0x7f680406ff20 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6804150520 con 0x7f680415a410 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f680a2bf640 10 monclient(hunting): authenticate will time out at 1136.862583s 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f68037fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f68040c99a0 0x7f68041f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f68037fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f68040c99a0 0x7f68041f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:50678/0 (socket says 10.20.193.6:50678) 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f68037fe640 1 -- 10.20.193.6:0/3520177637 learned_addr learned my addr 10.20.193.6:0/3520177637 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:22.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f68037fe640 10 monclient(hunting): get_auth_request con 0x7f68040c99a0 auth_method 0 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f68037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f68037fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f68037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.135+0000 7f6803fff640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f680406ff20 0x7f6804156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680415a410 0x7f680406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6803fff640 10 monclient(hunting): get_auth_request con 0x7f680406ff20 auth_method 0 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6803fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6803fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6803fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 10 monclient(hunting): get_auth_request con 0x7f680415a410 auth_method 0 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:22.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f68037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f68037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f68037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6803fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6803fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6803fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.136+0000 7f6808835640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 10 monclient(hunting): handle_auth_done global_id 4674 payload 824 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 1 -- 10.20.193.6:0/3520177637 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680415a410 msgr2=0x7f680406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680415a410 0x7f680406e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 1 -- 10.20.193.6:0/3520177637 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f680406ff20 msgr2=0x7f6804156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f680406ff20 0x7f6804156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 10 monclient: _finish_hunting 0 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 1 monclient: found mon.noname-a 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.137+0000 7f68037fe640 1 -- 10.20.193.6:0/3520177637 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6804156c70 con 0x7f68040c99a0 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.138+0000 7f68037fe640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f68040c99a0 0x7f68041f6430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f67f400ec10 tx=0x7f67f400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.138+0000 7f6800ff9640 10 client.?.objecter ms_handle_connect 0x7f68040c99a0 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.138+0000 7f6800ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 1 -- 10.20.193.6:0/3520177637 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4017070 con 0x7f68040c99a0 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:22.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: dump: 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: _finish_auth 0 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:24:22.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 5 monclient: authenticate success, global_id 4674 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 1 -- 10.20.193.6:0/3520177637 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f400cc40 con 0x7f68040c99a0 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 monclient: _renew_subs 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68041553b0 con 0x7f68040c99a0 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 1 -- 10.20.193.6:0/3520177637 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4013470 con 0x7f68040c99a0 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f68027fc640 4 set_mon_vals no callback set 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f6800ff9640 10 monclient: dump: 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:22.155 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 client.4674.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 monclient: _renew_subs 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6804155970 con 0x7f68040c99a0 2026-02-22T23:24:22.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 1 librados: init done 2026-02-22T23:24:22.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:22.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:22.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:22.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.140+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6804070300 con 0x7f68040c99a0 2026-02-22T23:24:22.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.151+0000 7f6800ff9640 1 -- 10.20.193.6:0/3520177637 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f67f4013630 con 0x7f68040c99a0 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.152+0000 7f6800ff9640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67d4078e00 0x7f67d407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.152+0000 7f6803fff640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67d4078e00 0x7f67d407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.152+0000 7f6803fff640 10 monclient: get_auth_request con 0x7f67d4078e00 auth_method 0 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.153+0000 7f6803fff640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67d4078e00 0x7f67d407b2c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67f0039000 tx=0x7f67f0033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.153+0000 7f6800ff9640 10 client.4674.objecter ms_handle_connect 0x7f67d4078e00 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.153+0000 7f6800ff9640 1 -- 10.20.193.6:0/3520177637 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f67f405e540 con 0x7f68040c99a0 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.153+0000 7f6800ff9640 10 client.4674.objecter ms_dispatch2 0x7f6804000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.153+0000 7f6800ff9640 3 client.4674.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.153+0000 7f6800ff9640 3 client.4674.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:22.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.153+0000 7f6800ff9640 20 client.4674.objecter dump_active .. 0 homeless 2026-02-22T23:24:22.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.249+0000 7f6800ff9640 1 -- 10.20.193.6:0/3520177637 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f67f4062500 con 0x7f68040c99a0 2026-02-22T23:24:22.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.249+0000 7f6800ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:22.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.249+0000 7f6800ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:22.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.350+0000 7f680a2bf640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:22.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.350+0000 7f680a2bf640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:22.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.350+0000 7f680a2bf640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:22.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.350+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f68041587f0 con 0x7f68040c99a0 2026-02-22T23:24:22.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.353+0000 7f6800ff9640 1 -- 10.20.193.6:0/3520177637 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f67f4066800 con 0x7f68040c99a0 2026-02-22T23:24:22.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.353+0000 7f6800ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:22.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.353+0000 7f6800ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:22.355 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:22.355 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 10 librados: watch_flush enter 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 10 librados: watch_flush exit 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 20 client.4674.objecter shutdown clearing up homeless session... 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 10 client.4674.objecter successfully canceled tick 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67d4078e00 msgr2=0x7f67d407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67d4078e00 0x7f67d407b2c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f67f0039000 tx=0x7f67f0033000 comp rx=0 tx=0).stop 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 10 monclient: shutdown 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:22.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f68040c99a0 msgr2=0x7f68041f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f68040c99a0 0x7f68041f6430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f67f400ec10 tx=0x7f67f400c520 comp rx=0 tx=0).stop 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 shutdown_connections 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67d4078e00 0x7f67d407b2c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f680415a410 0x7f680406e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f68040c99a0 0x7f68041f6430 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 --2- 10.20.193.6:0/3520177637 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f680406ff20 0x7f6804156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 >> 10.20.193.6:0/3520177637 conn(0x7f68040cd780 msgr2=0x7f680414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 shutdown_connections 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 -- 10.20.193.6:0/3520177637 wait complete. 2026-02-22T23:24:22.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:22.354+0000 7f680a2bf640 1 librados: shutdown 2026-02-22T23:24:25.365 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:25.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.433+0000 7f7accdeb640 10 monclient: get_monmap_and_config 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.433+0000 7f7accdeb640 10 monclient: build_initial_monmap 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.433+0000 7f7accdeb640 10 monclient: monmap: 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:25.435612+0000 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:25.435612+0000 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:25.437 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 1 Processor -- start 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 1 -- start start 2026-02-22T23:24:25.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient: init 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient: _add_conns ranks=[1,4,5,2,3,0] 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7ac80c4910 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7ac815be80 0x7f7ac80c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient(hunting): picked mon.noname-b con 0x7f7ac815be80 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f7ac815ced0 0x7f7ac81533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient(hunting): picked mon.noname-c con 0x7f7ac815ced0 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:25.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac815d360 con 0x7f7ac80c4910 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac815d630 con 0x7f7ac815be80 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac815d910 con 0x7f7ac815ced0 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.434+0000 7f7accdeb640 10 monclient(hunting): authenticate will time out at 1140.162972s 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7ac815be80 0x7f7ac80c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac7fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f7ac815ced0 0x7f7ac81533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac6ffd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f7ac815be80 0x7f7ac80c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:57540/0 (socket says 10.20.193.6:57540) 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac6ffd640 1 -- 10.20.193.6:0/3010654760 learned_addr learned my addr 10.20.193.6:0/3010654760 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac6ffd640 10 monclient(hunting): get_auth_request con 0x7f7ac815be80 auth_method 0 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:25.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac7fff640 10 monclient(hunting): get_auth_request con 0x7f7ac815ced0 auth_method 0 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.435+0000 7f7ac7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.436+0000 7f7ac7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.436+0000 7f7ac7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.436+0000 7f7ac7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.436+0000 7f7ac6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.436+0000 7f7ac6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.436+0000 7f7ac6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 10 monclient(hunting): handle_auth_done global_id 4766 payload 274 2026-02-22T23:24:25.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 1 -- 10.20.193.6:0/3010654760 >> v2:10.20.193.40:3300/0 conn(0x7f7ac815be80 msgr2=0x7f7ac80c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 1 --2- 10.20.193.6:0/3010654760 >> v2:10.20.193.40:3300/0 conn(0x7f7ac815be80 0x7f7ac80c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 1 -- 10.20.193.6:0/3010654760 >> v1:10.20.193.40:6789/0 conn(0x7f7ac80c4910 legacy=0x7f7ac815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 1 monclient: found mon.noname-c 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.437+0000 7f7ac7fff640 1 -- 10.20.193.6:0/3010654760 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac815dbf0 con 0x7f7ac815ced0 2026-02-22T23:24:25.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.438+0000 7f7ac7fff640 1 --2- 10.20.193.6:0/3010654760 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815ced0 0x7f7ac81533a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8074540 tx=0x7f7abc02f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86c3b52f1a80d7a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:25.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 1 -- 10.20.193.6:0/3010654760 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc03c070 con 0x7f7ac815ced0 2026-02-22T23:24:25.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:25.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:25.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: dump: 2026-02-22T23:24:25.451 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:25.451 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: _finish_auth 0 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:24:25.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7accdeb640 5 monclient: authenticate success, global_id 4766 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 1 -- 10.20.193.6:0/3010654760 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7abc037440 con 0x7f7ac815ced0 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7accdeb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 1 -- 10.20.193.6:0/3010654760 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc037810 con 0x7f7ac815ced0 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7ac67fc640 10 monclient: dump: 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.453 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7accdeb640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.440+0000 7f7accdeb640 4 set_mon_vals no callback set 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 10 monclient: shutdown 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 -- 10.20.193.6:0/3010654760 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815ced0 msgr2=0x7f7ac81533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 --2- 10.20.193.6:0/3010654760 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815ced0 0x7f7ac81533a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8074540 tx=0x7f7abc02f1c0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 -- 10.20.193.6:0/3010654760 shutdown_connections 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 --2- 10.20.193.6:0/3010654760 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815ced0 0x7f7ac81533a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 --2- 10.20.193.6:0/3010654760 >> v2:10.20.193.40:3300/0 conn(0x7f7ac815be80 0x7f7ac80c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 -- 10.20.193.6:0/3010654760 >> 10.20.193.6:0/3010654760 conn(0x7f7ac80cd480 msgr2=0x7f7ac80cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 -- 10.20.193.6:0/3010654760 shutdown_connections 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 1 -- 10.20.193.6:0/3010654760 wait complete. 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 10 monclient: build_initial_monmap 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.441+0000 7f7accdeb640 10 monclient: monmap: 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:25.455 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 librados: starting msgr at 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 librados: starting objecter 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 Processor -- start 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 -- start start 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 librados: setting wanted keys 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 librados: calling monclient init 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient: init 2026-02-22T23:24:25.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ac80c4910 0x7f7ac8150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): picked mon.noname-a con 0x7f7ac80c4910 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815be80 0x7f7ac81458e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): picked mon.noname-c con 0x7f7ac815be80 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ac815ced0 0x7f7ac80c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): picked mon.noname-b con 0x7f7ac815ced0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac81538e0 con 0x7f7ac80c4910 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac815b250 con 0x7f7ac815ced0 2026-02-22T23:24:25.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8153a60 con 0x7f7ac815be80 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7accdeb640 10 monclient(hunting): authenticate will time out at 1140.170665s 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.442+0000 7f7ac6ffd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815be80 0x7f7ac81458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ac80c4910 0x7f7ac8150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815be80 0x7f7ac81458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:39364/0 (socket says 10.20.193.6:39364) 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 1 -- 10.20.193.6:0/1987258059 learned_addr learned my addr 10.20.193.6:0/1987258059 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 10 monclient(hunting): get_auth_request con 0x7f7ac815be80 auth_method 0 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:25.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ac815ced0 0x7f7ac80c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 10 monclient(hunting): get_auth_request con 0x7f7ac80c4910 auth_method 0 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 10 monclient(hunting): get_auth_request con 0x7f7ac815ced0 auth_method 0 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:25.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.443+0000 7f7ac7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 10 monclient(hunting): handle_auth_done global_id 4769 payload 824 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 1 -- 10.20.193.6:0/1987258059 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ac815ced0 msgr2=0x7f7ac80c0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ac815ced0 0x7f7ac80c0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 1 -- 10.20.193.6:0/1987258059 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ac80c4910 msgr2=0x7f7ac8150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ac80c4910 0x7f7ac8150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 1 monclient: found mon.noname-c 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:25.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.444+0000 7f7ac6ffd640 1 -- 10.20.193.6:0/1987258059 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac80c1b50 con 0x7f7ac815be80 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.445+0000 7f7ac6ffd640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815be80 0x7f7ac81458e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e930 tx=0x7f7ab800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.445+0000 7f7aa3fff640 10 client.?.objecter ms_handle_connect 0x7f7ac815be80 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.445+0000 7f7aa3fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7aa3fff640 1 -- 10.20.193.6:0/1987258059 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8017070 con 0x7f7ac815be80 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7aa3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7aa3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7aa3fff640 10 monclient: dump: 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.461 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7aa3fff640 10 monclient: _finish_auth 0 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7aa3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7aa3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 5 monclient: authenticate success, global_id 4769 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 10 monclient: _renew_subs 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:25.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac80c1e30 con 0x7f7ac815be80 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 10 client.4769.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 10 monclient: _renew_subs 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac80bac40 con 0x7f7ac815be80 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.446+0000 7f7accdeb640 1 librados: init done 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7accdeb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7accdeb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7accdeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac80ba780 con 0x7f7ac815be80 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7aa3fff640 1 -- 10.20.193.6:0/1987258059 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab8005a90 con 0x7f7ac815be80 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7aa3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7aa3fff640 1 -- 10.20.193.6:0/1987258059 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8021060 con 0x7f7ac815be80 2026-02-22T23:24:25.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7aa3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7ac5ffb640 4 set_mon_vals no callback set 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7aa3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.447+0000 7f7aa3fff640 10 monclient: dump: 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:25.464 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:25.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.551+0000 7f7aa3fff640 1 -- 10.20.193.6:0/1987258059 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7ab8024060 con 0x7f7ac815be80 2026-02-22T23:24:25.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.551+0000 7f7aa3fff640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7a98078e00 0x7f7a9807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:25.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.551+0000 7f7ac77fe640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7a98078e00 0x7f7a9807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:25.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.551+0000 7f7ac77fe640 10 monclient: get_auth_request con 0x7f7a98078e00 auth_method 0 2026-02-22T23:24:25.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.552+0000 7f7ac77fe640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7a98078e00 0x7f7a9807b2c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40051f0 tx=0x7f7ab4007f30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:25.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.552+0000 7f7aa3fff640 10 client.4769.objecter ms_handle_connect 0x7f7a98078e00 2026-02-22T23:24:25.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.552+0000 7f7aa3fff640 1 -- 10.20.193.6:0/1987258059 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f7ab805e140 con 0x7f7ac815be80 2026-02-22T23:24:25.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.552+0000 7f7aa3fff640 10 client.4769.objecter ms_dispatch2 0x7f7ac8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:25.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.552+0000 7f7aa3fff640 3 client.4769.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:25.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.552+0000 7f7aa3fff640 3 client.4769.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:25.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.553+0000 7f7aa3fff640 20 client.4769.objecter dump_active .. 0 homeless 2026-02-22T23:24:25.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.559+0000 7f7aa3fff640 1 -- 10.20.193.6:0/1987258059 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7ac80ba780 con 0x7f7ac815be80 2026-02-22T23:24:25.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.559+0000 7f7aa3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:25.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.559+0000 7f7aa3fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:25.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.663+0000 7f7accdeb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:25.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.663+0000 7f7accdeb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:25.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.663+0000 7f7accdeb640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:25.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.663+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7ac80c4cf0 con 0x7f7ac815be80 2026-02-22T23:24:25.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.666+0000 7f7aa3fff640 1 -- 10.20.193.6:0/1987258059 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7ab8062100 con 0x7f7ac815be80 2026-02-22T23:24:25.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.666+0000 7f7aa3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:25.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.666+0000 7f7aa3fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:25.669 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:25.669 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:25.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 10 librados: watch_flush enter 2026-02-22T23:24:25.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 10 librados: watch_flush exit 2026-02-22T23:24:25.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 20 client.4769.objecter shutdown clearing up homeless session... 2026-02-22T23:24:25.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 10 client.4769.objecter successfully canceled tick 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7a98078e00 msgr2=0x7f7a9807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7a98078e00 0x7f7a9807b2c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40051f0 tx=0x7f7ab4007f30 comp rx=0 tx=0).stop 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 10 monclient: shutdown 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815be80 msgr2=0x7f7ac81458e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815be80 0x7f7ac81458e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e930 tx=0x7f7ab800ee00 comp rx=0 tx=0).stop 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 shutdown_connections 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7a98078e00 0x7f7a9807b2c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7ac815ced0 0x7f7ac80c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7ac815be80 0x7f7ac81458e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 --2- 10.20.193.6:0/1987258059 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7ac80c4910 0x7f7ac8150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:25.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 >> 10.20.193.6:0/1987258059 conn(0x7f7ac80cd480 msgr2=0x7f7ac8149f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:25.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 shutdown_connections 2026-02-22T23:24:25.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 -- 10.20.193.6:0/1987258059 wait complete. 2026-02-22T23:24:25.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:25.668+0000 7f7accdeb640 1 librados: shutdown 2026-02-22T23:24:28.680 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.748+0000 7ff5f7914640 10 monclient: get_monmap_and_config 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.748+0000 7ff5f7914640 10 monclient: build_initial_monmap 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.748+0000 7ff5f7914640 10 monclient: monmap: 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:28.750157+0000 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:28.750157+0000 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:28.751 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:28.752 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:28.752 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:28.752 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:28.752 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:28.752 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:28.752 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.748+0000 7ff5f7914640 1 Processor -- start 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 1 -- start start 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient: init 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient: _add_conns ranks=[2,0,4,3,5,1] 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff5f006ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff5f00c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7ff5f015a5e0 0x7ff5f0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:28.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient(hunting): picked mon.noname-b con 0x7ff5f015a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f00c7e90 con 0x7ff5f00c99a0 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f00c8170 con 0x7ff5f006ff20 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f0159a40 con 0x7ff5f015a5e0 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f7914640 10 monclient(hunting): authenticate will time out at 1143.477535s 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f4e88640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7ff5f00c99a0 0x7ff5f014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:42286/0 (socket says 10.20.193.6:42286) 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f4e88640 1 -- 10.20.193.6:0/3784775576 learned_addr learned my addr 10.20.193.6:0/3784775576 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.749+0000 7ff5f5e8a640 1 --2- 10.20.193.6:0/3784775576 >> v2:10.20.193.40:3300/0 conn(0x7ff5f015a5e0 0x7ff5f0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.750+0000 7ff5f5e8a640 10 monclient(hunting): get_auth_request con 0x7ff5f015a5e0 auth_method 0 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.750+0000 7ff5f5e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:28.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.750+0000 7ff5f5e8a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.750+0000 7ff5f5e8a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.750+0000 7ff5f5e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.750+0000 7ff5f5e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.750+0000 7ff5f5e8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 1 -- 10.20.193.6:0/3784775576 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642409741 0 0) 0x7ff5f00c7e90 con 0x7ff5f00c99a0 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): _negotiate 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): my global_id is 4680 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 1 -- 10.20.193.6:0/3784775576 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5d8003620 con 0x7ff5f00c99a0 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 1 -- 10.20.193.6:0/3784775576 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840492971 0 0) 0x7ff5f00c8170 con 0x7ff5f006ff20 2026-02-22T23:24:28.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): _negotiate 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 10 monclient(hunting): my global_id is 4772 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5dffff640 1 -- 10.20.193.6:0/3784775576 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f00c7e90 con 0x7ff5f006ff20 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 10 monclient(hunting): handle_auth_done global_id 4693 payload 274 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 1 -- 10.20.193.6:0/3784775576 >> v1:10.20.193.80:6789/0 conn(0x7ff5f006ff20 legacy=0x7ff5f0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 1 -- 10.20.193.6:0/3784775576 >> v1:10.20.193.6:6789/0 conn(0x7ff5f00c99a0 legacy=0x7ff5f014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 10 monclient: _finish_hunting 0 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 1 monclient: found mon.noname-b 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.751+0000 7ff5f5e8a640 1 -- 10.20.193.6:0/3784775576 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f00c8450 con 0x7ff5f015a5e0 2026-02-22T23:24:28.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.752+0000 7ff5f5e8a640 1 --2- 10.20.193.6:0/3784775576 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 0x7ff5f0070300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009dc0 tx=0x7ff5ec02f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f477f2b00be14c1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:28.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 1 -- 10.20.193.6:0/3784775576 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec03c070 con 0x7ff5f015a5e0 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: dump: 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:28.765 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: _finish_auth 0 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 1 -- 10.20.193.6:0/3784775576 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec037440 con 0x7ff5f015a5e0 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 5 monclient: authenticate success, global_id 4693 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 1 -- 10.20.193.6:0/3784775576 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec037810 con 0x7ff5f015a5e0 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 4 set_mon_vals no callback set 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5dffff640 10 monclient: dump: 2026-02-22T23:24:28.766 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 10 monclient: shutdown 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 1 -- 10.20.193.6:0/3784775576 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 msgr2=0x7ff5f0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 1 --2- 10.20.193.6:0/3784775576 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 0x7ff5f0070300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009dc0 tx=0x7ff5ec02f280 comp rx=0 tx=0).stop 2026-02-22T23:24:28.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 1 -- 10.20.193.6:0/3784775576 shutdown_connections 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 1 --2- 10.20.193.6:0/3784775576 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 0x7ff5f0070300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 1 -- 10.20.193.6:0/3784775576 >> 10.20.193.6:0/3784775576 conn(0x7ff5f00cd780 msgr2=0x7ff5f00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 1 -- 10.20.193.6:0/3784775576 shutdown_connections 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 1 -- 10.20.193.6:0/3784775576 wait complete. 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 10 monclient: build_initial_monmap 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.754+0000 7ff5f7914640 10 monclient: monmap: 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:28.768 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 librados: starting msgr at 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 librados: starting objecter 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 Processor -- start 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 -- start start 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 librados: setting wanted keys 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 librados: calling monclient init 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient: init 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5f006ff20 0x7ff5f00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): picked mon.noname-c con 0x7ff5f006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5f00c99a0 0x7ff5f01f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): picked mon.noname-a con 0x7ff5f00c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 0x7ff5f01f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:28.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): picked mon.noname-b con 0x7ff5f015a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0150520 con 0x7ff5f00c99a0 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f00c8a10 con 0x7ff5f015a5e0 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01506a0 con 0x7ff5f006ff20 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f7914640 10 monclient(hunting): authenticate will time out at 1143.483636s 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f4e88640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5f00c99a0 0x7ff5f01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f5e8a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 0x7ff5f01f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f4e88640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5f00c99a0 0x7ff5f01f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:46686/0 (socket says 10.20.193.6:46686) 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f4e88640 1 -- 10.20.193.6:0/440862588 learned_addr learned my addr 10.20.193.6:0/440862588 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f4e88640 10 monclient(hunting): get_auth_request con 0x7ff5f00c99a0 auth_method 0 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.755+0000 7ff5f4e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:28.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f4e88640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f4e88640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5f006ff20 0x7ff5f00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5e8a640 10 monclient(hunting): get_auth_request con 0x7ff5f015a5e0 auth_method 0 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5e8a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5e8a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 10 monclient(hunting): get_auth_request con 0x7ff5f006ff20 auth_method 0 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f4e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f4e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f4e88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:28.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5e8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.756+0000 7ff5f5689640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 10 monclient(hunting): handle_auth_done global_id 4683 payload 824 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 1 -- 10.20.193.6:0/440862588 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5f006ff20 msgr2=0x7ff5f00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5f006ff20 0x7ff5f00cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 1 -- 10.20.193.6:0/440862588 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 msgr2=0x7ff5f01f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 0x7ff5f01f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 10 monclient: _finish_hunting 0 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 1 monclient: found mon.noname-a 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.757+0000 7ff5f4e88640 1 -- 10.20.193.6:0/440862588 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f00bd2b0 con 0x7ff5f00c99a0 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.758+0000 7ff5f4e88640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5f00c99a0 0x7ff5f01f1570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c730 tx=0x7ff5e400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.758+0000 7ff5ddffb640 10 client.?.objecter ms_handle_connect 0x7ff5f00c99a0 2026-02-22T23:24:28.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.758+0000 7ff5ddffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 1 -- 10.20.193.6:0/440862588 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4017070 con 0x7ff5f00c99a0 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: dump: 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:28.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: _finish_auth 0 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: _check_auth_tickets 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 1 -- 10.20.193.6:0/440862588 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e4005bd0 con 0x7ff5f00c99a0 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 5 monclient: authenticate success, global_id 4683 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 monclient: _renew_subs 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f01f4a70 con 0x7ff5f00c99a0 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 1 -- 10.20.193.6:0/440862588 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4007070 con 0x7ff5f00c99a0 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5ddffb640 10 monclient: dump: 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.774 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5df7fe640 4 set_mon_vals no callback set 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 client.4683.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 monclient: _renew_subs 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f01f5030 con 0x7ff5f00c99a0 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 1 librados: init done 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:28.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.760+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f0070300 con 0x7ff5f00c99a0 2026-02-22T23:24:28.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.772+0000 7ff5ddffb640 1 -- 10.20.193.6:0/440862588 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff5e4007210 con 0x7ff5f00c99a0 2026-02-22T23:24:28.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.773+0000 7ff5ddffb640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5c0078e50 0x7ff5c007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:28.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.773+0000 7ff5f5689640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5c0078e50 0x7ff5c007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:28.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.773+0000 7ff5f5689640 10 monclient: get_auth_request con 0x7ff5c0078e50 auth_method 0 2026-02-22T23:24:28.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.774+0000 7ff5f5689640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5c0078e50 0x7ff5c007b310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0002410 tx=0x7ff5e0003d50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:28.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.774+0000 7ff5ddffb640 10 client.4683.objecter ms_handle_connect 0x7ff5c0078e50 2026-02-22T23:24:28.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 1 -- 10.20.193.6:0/440862588 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7ff5e405d4e0 con 0x7ff5f00c99a0 2026-02-22T23:24:28.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 10 client.4683.objecter ms_dispatch2 0x7ff5f0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:28.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 3 client.4683.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:28.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 3 client.4683.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:28.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 20 client.4683.objecter dump_active .. 0 homeless 2026-02-22T23:24:28.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 1 -- 10.20.193.6:0/440862588 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff5e4021020 con 0x7ff5f00c99a0 2026-02-22T23:24:28.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:28.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.874+0000 7ff5ddffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:28.976 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.974+0000 7ff5f7914640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:28.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.974+0000 7ff5f7914640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:28.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.974+0000 7ff5f7914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:28.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.974+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff5f00c4a10 con 0x7ff5f00c99a0 2026-02-22T23:24:28.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.976+0000 7ff5ddffb640 1 -- 10.20.193.6:0/440862588 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff5e40614a0 con 0x7ff5f00c99a0 2026-02-22T23:24:28.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.976+0000 7ff5ddffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:28.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.976+0000 7ff5ddffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:28.979 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:28.979 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:28.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 10 librados: watch_flush enter 2026-02-22T23:24:28.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 10 librados: watch_flush exit 2026-02-22T23:24:28.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 20 client.4683.objecter shutdown clearing up homeless session... 2026-02-22T23:24:28.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 10 client.4683.objecter successfully canceled tick 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5c0078e50 msgr2=0x7ff5c007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5c0078e50 0x7ff5c007b310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0002410 tx=0x7ff5e0003d50 comp rx=0 tx=0).stop 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 10 monclient: shutdown 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5f00c99a0 msgr2=0x7ff5f01f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5f00c99a0 0x7ff5f01f1570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c730 tx=0x7ff5e400cc00 comp rx=0 tx=0).stop 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 shutdown_connections 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff5c0078e50 0x7ff5c007b310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5f015a5e0 0x7ff5f01f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5f00c99a0 0x7ff5f01f1570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:28.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 --2- 10.20.193.6:0/440862588 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5f006ff20 0x7ff5f00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:28.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 >> 10.20.193.6:0/440862588 conn(0x7ff5f00cd780 msgr2=0x7ff5f014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:28.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.977+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 shutdown_connections 2026-02-22T23:24:28.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.978+0000 7ff5f7914640 1 -- 10.20.193.6:0/440862588 wait complete. 2026-02-22T23:24:28.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:28.978+0000 7ff5f7914640 1 librados: shutdown 2026-02-22T23:24:31.989 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.054+0000 7f3277ca6640 10 monclient: get_monmap_and_config 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.054+0000 7f3277ca6640 10 monclient: build_initial_monmap 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.054+0000 7f3277ca6640 10 monclient: monmap: 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:32.056395+0000 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:32.056395+0000 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 1 Processor -- start 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 1 -- start start 2026-02-22T23:24:32.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient: init 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient: _add_conns ranks=[1,2,4,5,0,3] 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3270155ef0 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f32701562d0 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f32700c9790 0x7f327015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient(hunting): picked mon.noname-b con 0x7f32700c9790 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32700ce3b0 con 0x7f3270155ef0 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32700ce690 con 0x7f32701562d0 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f327015cdb0 con 0x7f32700c9790 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.055+0000 7f3277ca6640 10 monclient(hunting): authenticate will time out at 1146.783906s 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.056+0000 7f327521a640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f32701562d0 0x7f32700c9080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:50740/0 (socket says 10.20.193.6:50740) 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.056+0000 7f327521a640 1 -- 10.20.193.6:0/1288417840 learned_addr learned my addr 10.20.193.6:0/1288417840 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:32.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.056+0000 7f327621c640 1 --2- 10.20.193.6:0/1288417840 >> v2:10.20.193.40:3300/0 conn(0x7f32700c9790 0x7f327015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:32.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f327621c640 10 monclient(hunting): get_auth_request con 0x7f32700c9790 auth_method 0 2026-02-22T23:24:32.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f327621c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:32.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f327621c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:32.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f327621c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:32.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f3274a19640 1 -- 10.20.193.6:0/1288417840 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2150792621 0 0) 0x7f32700ce690 con 0x7f32701562d0 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f3274a19640 10 monclient(hunting): _negotiate 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f3274a19640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f3274a19640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f3274a19640 10 monclient(hunting): my global_id is 4778 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.057+0000 7f3274a19640 1 -- 10.20.193.6:0/1288417840 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3258003620 con 0x7f32701562d0 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f3274a19640 1 -- 10.20.193.6:0/1288417840 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379230537 0 0) 0x7f32700ce3b0 con 0x7f3270155ef0 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f3274a19640 10 monclient(hunting): _negotiate 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f3274a19640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f3274a19640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f3274a19640 10 monclient(hunting): my global_id is 4699 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f3274a19640 1 -- 10.20.193.6:0/1288417840 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32700ce690 con 0x7f3270155ef0 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f327621c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f327621c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:32.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.058+0000 7f327621c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 10 monclient(hunting): handle_auth_done global_id 4702 payload 274 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 1 -- 10.20.193.6:0/1288417840 >> v1:10.20.193.80:6789/0 conn(0x7f32701562d0 legacy=0x7f32700c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 1 -- 10.20.193.6:0/1288417840 >> v1:10.20.193.40:6789/0 conn(0x7f3270155ef0 legacy=0x7f32700cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 10 monclient: _finish_hunting 0 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 1 monclient: found mon.noname-b 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:24:32.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.059+0000 7f327621c640 1 -- 10.20.193.6:0/1288417840 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32700ce970 con 0x7f32700c9790 2026-02-22T23:24:32.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.062+0000 7f327621c640 1 --2- 10.20.193.6:0/1288417840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32700c9790 0x7f327015c870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d90 tx=0x7f326c02f2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd430a0f43c480a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 1 -- 10.20.193.6:0/1288417840 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c03c070 con 0x7f32700c9790 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: dump: 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: _finish_auth 0 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: _check_auth_tickets 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 1 -- 10.20.193.6:0/1288417840 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c037440 con 0x7f32700c9790 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 5 monclient: authenticate success, global_id 4702 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 1 -- 10.20.193.6:0/1288417840 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c037810 con 0x7f32700c9790 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 4 set_mon_vals no callback set 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3274a19640 10 monclient: dump: 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.066 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 10 monclient: shutdown 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 1 -- 10.20.193.6:0/1288417840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32700c9790 msgr2=0x7f327015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 1 --2- 10.20.193.6:0/1288417840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32700c9790 0x7f327015c870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f326c009d90 tx=0x7f326c02f2c0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 1 -- 10.20.193.6:0/1288417840 shutdown_connections 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.063+0000 7f3277ca6640 1 --2- 10.20.193.6:0/1288417840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f32700c9790 0x7f327015c870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 -- 10.20.193.6:0/1288417840 >> 10.20.193.6:0/1288417840 conn(0x7f327006d8c0 msgr2=0x7f327006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:32.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 -- 10.20.193.6:0/1288417840 shutdown_connections 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 -- 10.20.193.6:0/1288417840 wait complete. 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 10 monclient: build_initial_monmap 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 10 monclient: monmap: 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 librados: starting msgr at 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 librados: starting objecter 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 Processor -- start 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 -- start start 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 librados: setting wanted keys 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 1 librados: calling monclient init 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.064+0000 7f3277ca6640 10 monclient: init 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f32700c9790 0x7f327014f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): picked mon.noname-c con 0x7f32700c9790 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3270155ef0 0x7f32701f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): picked mon.noname-b con 0x7f3270155ef0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32701562d0 0x7f327014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): picked mon.noname-a con 0x7f32701562d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:32.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f32700cef30 con 0x7f32701562d0 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f32700d2730 con 0x7f3270155ef0 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f32700cde80 con 0x7f32700c9790 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3277ca6640 10 monclient(hunting): authenticate will time out at 1146.793409s 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327621c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32701562d0 0x7f327014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327621c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32701562d0 0x7f327014d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:46690/0 (socket says 10.20.193.6:46690) 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327621c640 1 -- 10.20.193.6:0/3396023847 learned_addr learned my addr 10.20.193.6:0/3396023847 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327621c640 10 monclient(hunting): get_auth_request con 0x7f32701562d0 auth_method 0 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327621c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327621c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327621c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f327521a640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3270155ef0 0x7f32701f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.065+0000 7f3275a1b640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f32700c9790 0x7f327014f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327621c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327621c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327621c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327521a640 10 monclient(hunting): get_auth_request con 0x7f3270155ef0 auth_method 0 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327521a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327521a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:32.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327521a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f3275a1b640 10 monclient(hunting): get_auth_request con 0x7f32700c9790 auth_method 0 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f3275a1b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f3275a1b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f3275a1b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f3275a1b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f3275a1b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f3275a1b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327521a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327521a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.066+0000 7f327521a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 10 monclient(hunting): handle_auth_done global_id 4686 payload 824 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 1 -- 10.20.193.6:0/3396023847 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f32700c9790 msgr2=0x7f327014f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f32700c9790 0x7f327014f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 1 -- 10.20.193.6:0/3396023847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3270155ef0 msgr2=0x7f32701f3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3270155ef0 0x7f32701f3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 10 monclient: _finish_hunting 0 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 1 monclient: found mon.noname-a 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.067+0000 7f327621c640 1 -- 10.20.193.6:0/3396023847 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327014e080 con 0x7f32701562d0 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.068+0000 7f325e7fc640 10 client.?.objecter ms_handle_connect 0x7f32701562d0 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.068+0000 7f325e7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:32.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.068+0000 7f327621c640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32701562d0 0x7f327014d640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f326c009ec0 tx=0x7f326c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:32.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 1 -- 10.20.193.6:0/3396023847 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c03c050 con 0x7f32701562d0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: dump: 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: _finish_auth 0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 1 -- 10.20.193.6:0/3396023847 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c03d040 con 0x7f32701562d0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 1 -- 10.20.193.6:0/3396023847 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c041610 con 0x7f32701562d0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325e7fc640 10 monclient: dump: 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f325ffff640 4 set_mon_vals no callback set 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 5 monclient: authenticate success, global_id 4686 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 10 monclient: _renew_subs 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701f4dd0 con 0x7f32701562d0 2026-02-22T23:24:32.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 10 client.4686.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 10 monclient: _renew_subs 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701f5390 con 0x7f32701562d0 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.070+0000 7f3277ca6640 1 librados: init done 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.071+0000 7f3277ca6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.071+0000 7f3277ca6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.071+0000 7f3277ca6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:32.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.071+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327015c870 con 0x7f32701562d0 2026-02-22T23:24:32.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.082+0000 7f325e7fc640 1 -- 10.20.193.6:0/3396023847 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f326c0417b0 con 0x7f32701562d0 2026-02-22T23:24:32.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.082+0000 7f325e7fc640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3240078e00 0x7f324007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:32.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.083+0000 7f3275a1b640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3240078e00 0x7f324007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:32.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.083+0000 7f3275a1b640 10 monclient: get_auth_request con 0x7f3240078e00 auth_method 0 2026-02-22T23:24:32.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.083+0000 7f3275a1b640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3240078e00 0x7f324007b2c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3260007fd0 tx=0x7f3260003d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:32.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.083+0000 7f325e7fc640 10 client.4686.objecter ms_handle_connect 0x7f3240078e00 2026-02-22T23:24:32.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.085+0000 7f325e7fc640 1 -- 10.20.193.6:0/3396023847 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f326c0bd7d0 con 0x7f32701562d0 2026-02-22T23:24:32.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.085+0000 7f325e7fc640 10 client.4686.objecter ms_dispatch2 0x7f3270000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:32.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.085+0000 7f325e7fc640 3 client.4686.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:32.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.085+0000 7f325e7fc640 3 client.4686.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:32.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.085+0000 7f325e7fc640 20 client.4686.objecter dump_active .. 0 homeless 2026-02-22T23:24:32.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.183+0000 7f325e7fc640 1 -- 10.20.193.6:0/3396023847 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f326c087fa0 con 0x7f32701562d0 2026-02-22T23:24:32.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.183+0000 7f325e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:32.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.183+0000 7f325e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:32.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.282+0000 7f3277ca6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:32.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.282+0000 7f3277ca6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:32.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.282+0000 7f3277ca6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:32.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.282+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f32700caa30 con 0x7f32701562d0 2026-02-22T23:24:32.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.285+0000 7f325e7fc640 1 -- 10.20.193.6:0/3396023847 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f326c08c2a0 con 0x7f32701562d0 2026-02-22T23:24:32.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.285+0000 7f325e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:32.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.285+0000 7f325e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:32.287 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:32.288 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:32.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 10 librados: watch_flush enter 2026-02-22T23:24:32.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 10 librados: watch_flush exit 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 20 client.4686.objecter shutdown clearing up homeless session... 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 10 client.4686.objecter successfully canceled tick 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3240078e00 msgr2=0x7f324007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3240078e00 0x7f324007b2c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3260007fd0 tx=0x7f3260003d90 comp rx=0 tx=0).stop 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 10 monclient: shutdown 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32701562d0 msgr2=0x7f327014d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32701562d0 0x7f327014d640 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f326c009ec0 tx=0x7f326c004b40 comp rx=0 tx=0).stop 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 shutdown_connections 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3240078e00 0x7f324007b2c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f32701562d0 0x7f327014d640 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3270155ef0 0x7f32701f3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 --2- 10.20.193.6:0/3396023847 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f32700c9790 0x7f327014f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 >> 10.20.193.6:0/3396023847 conn(0x7f327006d8c0 msgr2=0x7f3270153a50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 shutdown_connections 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 -- 10.20.193.6:0/3396023847 wait complete. 2026-02-22T23:24:32.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:32.286+0000 7f3277ca6640 1 librados: shutdown 2026-02-22T23:24:32.961 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:17.211 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:32.988 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:17.235 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:32.990 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:17.201 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:32.991 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:17.240 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:32.991 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:17.237 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:32.993 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:17.204 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:33.000 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:17.247 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:33.003 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:17.253 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:33.003 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:17.215 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:35.298 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.364+0000 7f51121c3640 10 monclient: get_monmap_and_config 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.364+0000 7f51121c3640 10 monclient: build_initial_monmap 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.364+0000 7f51121c3640 10 monclient: monmap: 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:35.366278+0000 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:35.366278+0000 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:35.367 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 1 Processor -- start 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 1 -- start start 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient: init 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient: _add_conns ranks=[0,2,1,5,3,4] 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f510c06ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f510c0c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f510c15a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510c0c7e90 con 0x7f510c06ff20 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510c0c8170 con 0x7f510c15a5e0 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510c0c8450 con 0x7f510c0c99a0 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.365+0000 7f51121c3640 10 monclient(hunting): authenticate will time out at 1150.093739s 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.366+0000 7f510b7fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f510c06ff20 0x7f510c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:42296/0 (socket says 10.20.193.6:42296) 2026-02-22T23:24:35.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.366+0000 7f510b7fe640 1 -- 10.20.193.6:0/1226227168 learned_addr learned my addr 10.20.193.6:0/1226227168 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:35.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80912540 0 0) 0x7f510c0c8450 con 0x7f510c0c99a0 2026-02-22T23:24:35.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:24:35.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:35.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): my global_id is 4784 2026-02-22T23:24:35.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f0003620 con 0x7f510c0c99a0 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 914128200 0 0) 0x7f510c0c7e90 con 0x7f510c06ff20 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): my global_id is 4689 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f510c0c8450 con 0x7f510c06ff20 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1747523579 0 0) 0x7f510c0c8170 con 0x7f510c15a5e0 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 10 monclient(hunting): my global_id is 4708 2026-02-22T23:24:35.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.367+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f510c0c7e90 con 0x7f510c15a5e0 2026-02-22T23:24:35.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2807371254 0 0) 0x7f50f0003620 con 0x7f510c0c99a0 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 >> v1:10.20.193.40:6789/0 conn(0x7f510c15a5e0 legacy=0x7f510c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 >> v1:10.20.193.6:6789/0 conn(0x7f510c06ff20 legacy=0x7f510c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c0c8730 con 0x7f510c0c99a0 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f50fc002f00 con 0x7f510c0c99a0 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: dump: 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:24:35.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 >> v1:10.20.193.80:6789/0 conn(0x7f510c0c99a0 legacy=0x7f510c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0003fe0 0x7f50f0004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): picked mon.c con 0x7f50f0003fe0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f50f0004e40 0x7f50f00052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): picked mon.b con 0x7f50f0004e40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f50f0009ab0 0x7f50f000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): picked mon.a con 0x7f50f0009ab0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f50f000c570 con 0x7f50f0009ab0 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50f000c6f0 con 0x7f50f0004e40 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f50f000c8f0 con 0x7f50f0003fe0 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:24:35.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.369+0000 7f510b7fe640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0003fe0 0x7f50f0004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f50f0009ab0 0x7f50f000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f50f0004e40 0x7f50f00052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 10 monclient(hunting): get_auth_request con 0x7f50f0009ab0 auth_method 0 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 10 monclient(hunting): get_auth_request con 0x7f50f0004e40 auth_method 0 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510b7fe640 10 monclient(hunting): get_auth_request con 0x7f50f0003fe0 auth_method 0 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:35.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510bfff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.370+0000 7f510b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 10 monclient(hunting): handle_auth_done global_id 4784 payload 293 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 1 -- 10.20.193.6:0/1226227168 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0003fe0 msgr2=0x7f50f0004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0003fe0 0x7f50f0004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 1 -- 10.20.193.6:0/1226227168 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f50f0004e40 msgr2=0x7f50f00052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f50f0004e40 0x7f50f00052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 1 monclient: found mon.a 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 1 -- 10.20.193.6:0/1226227168 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f0003620 con 0x7f50f0009ab0 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 10 monclient: _finish_auth 0 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f51121c3640 5 monclient: authenticate success, global_id 4784 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f51121c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.371+0000 7f510bfff640 1 -- 10.20.193.6:0/1226227168 reap_dead start 2026-02-22T23:24:35.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.372+0000 7f510bfff640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f50f0009ab0 0x7f50f000bea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f510000c320 tx=0x7f510002f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40607a27fa911e9 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100003860 con 0x7f50f0009ab0 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 10 monclient: dump: 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:35.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100003c70 con 0x7f50f0009ab0 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 1 -- 10.20.193.6:0/1226227168 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100005310 con 0x7f50f0009ab0 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f510a7fc640 10 monclient: dump: 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:35.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 4 set_mon_vals no callback set 2026-02-22T23:24:35.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 10 monclient: shutdown 2026-02-22T23:24:35.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 1 -- 10.20.193.6:0/1226227168 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f50f0009ab0 msgr2=0x7f50f000bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f50f0009ab0 0x7f50f000bea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f510000c320 tx=0x7f510002f610 comp rx=0 tx=0).stop 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 1 -- 10.20.193.6:0/1226227168 shutdown_connections 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 1 --2- 10.20.193.6:0/1226227168 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f50f0009ab0 0x7f50f000bea0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 1 -- 10.20.193.6:0/1226227168 >> 10.20.193.6:0/1226227168 conn(0x7f510c0cd780 msgr2=0x7f510c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 1 -- 10.20.193.6:0/1226227168 shutdown_connections 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.374+0000 7f51121c3640 1 -- 10.20.193.6:0/1226227168 wait complete. 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient: build_initial_monmap 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient: monmap: 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 librados: starting msgr at 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 librados: starting objecter 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 Processor -- start 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 -- start start 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 librados: setting wanted keys 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 librados: calling monclient init 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient: init 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0009ab0 0x7f510c1490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:35.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): picked mon.noname-c con 0x7f50f0009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f510c1551e0 0x7f510c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): picked mon.noname-a con 0x7f510c1551e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f510c155660 0x7f510c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): picked mon.noname-b con 0x7f510c155660 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f510c070300 con 0x7f510c1551e0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f510c159a40 con 0x7f510c155660 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f510c070480 con 0x7f50f0009ab0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.375+0000 7f51121c3640 10 monclient(hunting): authenticate will time out at 1150.103892s 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f510c1551e0 0x7f510c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0009ab0 0x7f510c1490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f510c1551e0 0x7f510c14bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:46706/0 (socket says 10.20.193.6:46706) 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 1 -- 10.20.193.6:0/2349620223 learned_addr learned my addr 10.20.193.6:0/2349620223 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 10 monclient(hunting): get_auth_request con 0x7f510c1551e0 auth_method 0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f510c155660 0x7f510c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 10 monclient(hunting): get_auth_request con 0x7f50f0009ab0 auth_method 0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 10 monclient(hunting): get_auth_request con 0x7f510c155660 auth_method 0 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:35.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.376+0000 7f510bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 10 monclient(hunting): handle_auth_done global_id 4692 payload 824 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 1 -- 10.20.193.6:0/2349620223 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0009ab0 msgr2=0x7f510c1490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0009ab0 0x7f510c1490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 1 -- 10.20.193.6:0/2349620223 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f510c155660 msgr2=0x7f510c14c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f510c155660 0x7f510c14c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 10 monclient: _finish_hunting 0 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 1 monclient: found mon.noname-a 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:35.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:35.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.377+0000 7f510affd640 1 -- 10.20.193.6:0/2349620223 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c1f1ed0 con 0x7f510c1551e0 2026-02-22T23:24:35.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.378+0000 7f510affd640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f510c1551e0 0x7f510c14bbd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00f850 tx=0x7f50fc00fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:35.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.378+0000 7f50e7fff640 10 client.?.objecter ms_handle_connect 0x7f510c1551e0 2026-02-22T23:24:35.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.378+0000 7f50e7fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 1 -- 10.20.193.6:0/2349620223 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0052e0 con 0x7f510c1551e0 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: dump: 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: _finish_auth 0 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 1 -- 10.20.193.6:0/2349620223 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc018070 con 0x7f510c1551e0 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 1 -- 10.20.193.6:0/2349620223 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0036f0 con 0x7f510c1551e0 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f50e7fff640 10 monclient: dump: 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:35.383 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f5109ffb640 4 set_mon_vals no callback set 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 5 monclient: authenticate success, global_id 4692 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 10 monclient: _renew_subs 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c154440 con 0x7f510c1551e0 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 10 client.4692.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 10 monclient: _renew_subs 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c154700 con 0x7f510c1551e0 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.380+0000 7f51121c3640 1 librados: init done 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.381+0000 7f51121c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.381+0000 7f51121c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.381+0000 7f51121c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:35.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.381+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c159ed0 con 0x7f510c1551e0 2026-02-22T23:24:35.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.392+0000 7f50e7fff640 1 -- 10.20.193.6:0/2349620223 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f50fc003890 con 0x7f510c1551e0 2026-02-22T23:24:35.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.392+0000 7f50e7fff640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f50dc078e00 0x7f50dc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:35.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.392+0000 7f510b7fe640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f50dc078e00 0x7f50dc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:35.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.393+0000 7f510b7fe640 10 monclient: get_auth_request con 0x7f50dc078e00 auth_method 0 2026-02-22T23:24:35.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.393+0000 7f510b7fe640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f50dc078e00 0x7f50dc07b2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50f8004e80 tx=0x7f50f8007e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:35.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.393+0000 7f50e7fff640 10 client.4692.objecter ms_handle_connect 0x7f50dc078e00 2026-02-22T23:24:35.494 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 1 -- 10.20.193.6:0/2349620223 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f50fc05e2f0 con 0x7f510c1551e0 2026-02-22T23:24:35.494 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 10 client.4692.objecter ms_dispatch2 0x7f510c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:35.494 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 3 client.4692.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:35.494 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 3 client.4692.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:35.494 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 20 client.4692.objecter dump_active .. 0 homeless 2026-02-22T23:24:35.494 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 1 -- 10.20.193.6:0/2349620223 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f50fc0622b0 con 0x7f510c1551e0 2026-02-22T23:24:35.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:35.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.491+0000 7f50e7fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:35.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.593+0000 7f51121c3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:35.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.593+0000 7f51121c3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:35.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.593+0000 7f51121c3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:35.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.593+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f510c0c5cf0 con 0x7f510c1551e0 2026-02-22T23:24:35.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.595+0000 7f50e7fff640 1 -- 10.20.193.6:0/2349620223 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f50fc0665b0 con 0x7f510c1551e0 2026-02-22T23:24:35.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.595+0000 7f50e7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:35.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.595+0000 7f50e7fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:35.597 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:35.597 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.596+0000 7f51121c3640 10 librados: watch_flush enter 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.596+0000 7f51121c3640 10 librados: watch_flush exit 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.596+0000 7f51121c3640 20 client.4692.objecter shutdown clearing up homeless session... 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.596+0000 7f51121c3640 10 client.4692.objecter successfully canceled tick 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.596+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f50dc078e00 msgr2=0x7f50dc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f50dc078e00 0x7f50dc07b2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50f8004e80 tx=0x7f50f8007e90 comp rx=0 tx=0).stop 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 10 monclient: shutdown 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f510c1551e0 msgr2=0x7f510c14bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f510c1551e0 0x7f510c14bbd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00f850 tx=0x7f50fc00fd20 comp rx=0 tx=0).stop 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 shutdown_connections 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f50dc078e00 0x7f50dc07b2c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f510c155660 0x7f510c14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f510c1551e0 0x7f510c14bbd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 --2- 10.20.193.6:0/2349620223 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50f0009ab0 0x7f510c1490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 >> 10.20.193.6:0/2349620223 conn(0x7f510c0cd780 msgr2=0x7f510c0c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 shutdown_connections 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 -- 10.20.193.6:0/2349620223 wait complete. 2026-02-22T23:24:35.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:35.597+0000 7f51121c3640 1 librados: shutdown 2026-02-22T23:24:38.609 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:38.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.675+0000 7fcc428a9640 10 monclient: get_monmap_and_config 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.675+0000 7fcc428a9640 10 monclient: build_initial_monmap 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.675+0000 7fcc428a9640 10 monclient: monmap: 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:38.677162+0000 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:38.677162+0000 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:38.678 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:38.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.675+0000 7fcc428a9640 1 Processor -- start 2026-02-22T23:24:38.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 1 -- start start 2026-02-22T23:24:38.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient: init 2026-02-22T23:24:38.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:38.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient: _add_conns ranks=[5,2,3,1,0,4] 2026-02-22T23:24:38.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fcc3c06ff20 0x7fcc3c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:38.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient(hunting): picked mon.noname-c con 0x7fcc3c06ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcc3c15c970 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcc3c155eb0 0x7fcc3c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient(hunting): picked mon.noname-a con 0x7fcc3c155eb0 addr v2:10.20.193.6:3300/0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc3c0725b0 con 0x7fcc3c15c970 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c072890 con 0x7fcc3c155eb0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcc3c072b70 con 0x7fcc3c06ff20 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc428a9640 10 monclient(hunting): authenticate will time out at 1153.404543s 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.676+0000 7fcc40e1f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcc3c155eb0 0x7fcc3c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc3bfff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fcc3c06ff20 0x7fcc3c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcc3c155eb0 0x7fcc3c15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47368/0 (socket says 10.20.193.6:47368) 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 1 -- 10.20.193.6:0/1133551970 learned_addr learned my addr 10.20.193.6:0/1133551970 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 10 monclient(hunting): get_auth_request con 0x7fcc3c155eb0 auth_method 0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc40e1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc3bfff640 10 monclient(hunting): get_auth_request con 0x7fcc3c06ff20 auth_method 0 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc3bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:38.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.677+0000 7fcc3bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 10 monclient(hunting): handle_auth_done global_id 4695 payload 274 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 1 -- 10.20.193.6:0/1133551970 >> v2:10.20.193.80:3300/0 conn(0x7fcc3c06ff20 msgr2=0x7fcc3c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 1 --2- 10.20.193.6:0/1133551970 >> v2:10.20.193.80:3300/0 conn(0x7fcc3c06ff20 0x7fcc3c070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 1 -- 10.20.193.6:0/1133551970 >> v1:10.20.193.80:6789/0 conn(0x7fcc3c15c970 legacy=0x7fcc3c1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 10 monclient: _finish_hunting 0 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 1 monclient: found mon.noname-a 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.678+0000 7fcc40e1f640 1 -- 10.20.193.6:0/1133551970 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c072e50 con 0x7fcc3c155eb0 2026-02-22T23:24:38.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.679+0000 7fcc40e1f640 1 --2- 10.20.193.6:0/1133551970 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c155eb0 0x7fcc3c15d1c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c156590 tx=0x7fcc3002f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffccd0e4401d2813 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:38.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 1 -- 10.20.193.6:0/1133551970 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3002fd70 con 0x7fcc3c155eb0 2026-02-22T23:24:38.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: dump: 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: _finish_auth 0 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: _check_auth_tickets 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 1 -- 10.20.193.6:0/1133551970 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc300373f0 con 0x7fcc3c155eb0 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 5 monclient: authenticate success, global_id 4695 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 1 -- 10.20.193.6:0/1133551970 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc300377c0 con 0x7fcc3c155eb0 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc3affd640 10 monclient: dump: 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:38.684 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 4 set_mon_vals no callback set 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 10 monclient: shutdown 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 -- 10.20.193.6:0/1133551970 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c155eb0 msgr2=0x7fcc3c15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 --2- 10.20.193.6:0/1133551970 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c155eb0 0x7fcc3c15d1c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c156590 tx=0x7fcc3002f270 comp rx=0 tx=0).stop 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 -- 10.20.193.6:0/1133551970 shutdown_connections 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 --2- 10.20.193.6:0/1133551970 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c155eb0 0x7fcc3c15d1c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 --2- 10.20.193.6:0/1133551970 >> v2:10.20.193.80:3300/0 conn(0x7fcc3c06ff20 0x7fcc3c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 -- 10.20.193.6:0/1133551970 >> 10.20.193.6:0/1133551970 conn(0x7fcc3c0cd780 msgr2=0x7fcc3c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 -- 10.20.193.6:0/1133551970 shutdown_connections 2026-02-22T23:24:38.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.681+0000 7fcc428a9640 1 -- 10.20.193.6:0/1133551970 wait complete. 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient: build_initial_monmap 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient: monmap: 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 librados: starting msgr at 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 librados: starting objecter 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 Processor -- start 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 -- start start 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 librados: setting wanted keys 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 librados: calling monclient init 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient: init 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c06ff20 0x7fcc3c1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): picked mon.noname-a con 0x7fcc3c06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc3c155eb0 0x7fcc3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): picked mon.noname-c con 0x7fcc3c155eb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc3c15c970 0x7fcc3c14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): picked mon.noname-b con 0x7fcc3c15c970 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0737d0 con 0x7fcc3c06ff20 2026-02-22T23:24:38.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c073650 con 0x7fcc3c15c970 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c073950 con 0x7fcc3c155eb0 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc428a9640 10 monclient(hunting): authenticate will time out at 1153.410693s 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.682+0000 7fcc3bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c06ff20 0x7fcc3c1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c06ff20 0x7fcc3c1f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47382/0 (socket says 10.20.193.6:47382) 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 1 -- 10.20.193.6:0/2842020013 learned_addr learned my addr 10.20.193.6:0/2842020013 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 10 monclient(hunting): get_auth_request con 0x7fcc3c06ff20 auth_method 0 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc3c155eb0 0x7fcc3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc3c15c970 0x7fcc3c14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 10 monclient(hunting): get_auth_request con 0x7fcc3c155eb0 auth_method 0 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 10 monclient(hunting): get_auth_request con 0x7fcc3c15c970 auth_method 0 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc3b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:38.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.683+0000 7fcc40e1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 10 monclient(hunting): handle_auth_done global_id 4698 payload 824 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 1 -- 10.20.193.6:0/2842020013 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc3c155eb0 msgr2=0x7fcc3c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc3c155eb0 0x7fcc3c06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 1 -- 10.20.193.6:0/2842020013 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc3c15c970 msgr2=0x7fcc3c14ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc3c15c970 0x7fcc3c14ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 1 monclient: found mon.noname-a 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:38.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.684+0000 7fcc3bfff640 1 -- 10.20.193.6:0/2842020013 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c06da30 con 0x7fcc3c06ff20 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.685+0000 7fcc3bfff640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c06ff20 0x7fcc3c1f4b20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800e890 tx=0x7fcc2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.685+0000 7fcc38ff9640 10 client.?.objecter ms_handle_connect 0x7fcc3c06ff20 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.685+0000 7fcc38ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc38ff9640 1 -- 10.20.193.6:0/2842020013 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28017070 con 0x7fcc3c06ff20 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc38ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc38ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc38ff9640 10 monclient: dump: 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc38ff9640 10 monclient: _finish_auth 0 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc38ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc38ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 5 monclient: authenticate success, global_id 4698 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 10 monclient: _renew_subs 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c06dd10 con 0x7fcc3c06ff20 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 10 client.4698.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 10 monclient: _renew_subs 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:38.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c06e2d0 con 0x7fcc3c06ff20 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.686+0000 7fcc428a9640 1 librados: init done 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc428a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc428a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc428a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c070300 con 0x7fcc3c06ff20 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc38ff9640 1 -- 10.20.193.6:0/2842020013 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc28005a90 con 0x7fcc3c06ff20 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc38ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc38ff9640 1 -- 10.20.193.6:0/2842020013 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28021060 con 0x7fcc3c06ff20 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc38ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc3a7fc640 4 set_mon_vals no callback set 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc38ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.687+0000 7fcc38ff9640 10 monclient: dump: 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:38.691 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:38.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.788+0000 7fcc38ff9640 1 -- 10.20.193.6:0/2842020013 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fcc2800c940 con 0x7fcc3c06ff20 2026-02-22T23:24:38.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.788+0000 7fcc38ff9640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc0c078e00 0x7fcc0c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:38.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.788+0000 7fcc3b7fe640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc0c078e00 0x7fcc0c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:38.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.788+0000 7fcc3b7fe640 10 monclient: get_auth_request con 0x7fcc0c078e00 auth_method 0 2026-02-22T23:24:38.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.789+0000 7fcc3b7fe640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc0c078e00 0x7fcc0c07b2c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005110 tx=0x7fcc2c007f30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:38.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.789+0000 7fcc38ff9640 10 client.4698.objecter ms_handle_connect 0x7fcc0c078e00 2026-02-22T23:24:38.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 1 -- 10.20.193.6:0/2842020013 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fcc2805e3d0 con 0x7fcc3c06ff20 2026-02-22T23:24:38.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 10 client.4698.objecter ms_dispatch2 0x7fcc3c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:38.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 3 client.4698.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:38.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 3 client.4698.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:38.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 20 client.4698.objecter dump_active .. 0 homeless 2026-02-22T23:24:38.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 1 -- 10.20.193.6:0/2842020013 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcc28062390 con 0x7fcc3c06ff20 2026-02-22T23:24:38.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:38.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.796+0000 7fcc38ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:38.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.896+0000 7fcc428a9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:38.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.896+0000 7fcc428a9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:38.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.896+0000 7fcc428a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:38.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.896+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcc3c152930 con 0x7fcc3c06ff20 2026-02-22T23:24:38.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.898+0000 7fcc38ff9640 1 -- 10.20.193.6:0/2842020013 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fcc28066690 con 0x7fcc3c06ff20 2026-02-22T23:24:38.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.898+0000 7fcc38ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:38.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.898+0000 7fcc38ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:38.901 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:38.902 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:38.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 10 librados: watch_flush enter 2026-02-22T23:24:38.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 10 librados: watch_flush exit 2026-02-22T23:24:38.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 20 client.4698.objecter shutdown clearing up homeless session... 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 10 client.4698.objecter successfully canceled tick 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc0c078e00 msgr2=0x7fcc0c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc0c078e00 0x7fcc0c07b2c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005110 tx=0x7fcc2c007f30 comp rx=0 tx=0).stop 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 10 monclient: shutdown 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c06ff20 msgr2=0x7fcc3c1f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c06ff20 0x7fcc3c1f4b20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800e890 tx=0x7fcc2800ed60 comp rx=0 tx=0).stop 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 shutdown_connections 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc0c078e00 0x7fcc0c07b2c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc3c15c970 0x7fcc3c14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc3c155eb0 0x7fcc3c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 --2- 10.20.193.6:0/2842020013 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc3c06ff20 0x7fcc3c1f4b20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 >> 10.20.193.6:0/2842020013 conn(0x7fcc3c0cd780 msgr2=0x7fcc3c15d930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 shutdown_connections 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 -- 10.20.193.6:0/2842020013 wait complete. 2026-02-22T23:24:38.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:38.899+0000 7fcc428a9640 1 librados: shutdown 2026-02-22T23:24:41.911 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.976+0000 7fe9d36f5640 10 monclient: get_monmap_and_config 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.976+0000 7fe9d36f5640 10 monclient: build_initial_monmap 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.976+0000 7fe9d36f5640 10 monclient: monmap: 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:41.978507+0000 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:41.978507+0000 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.977+0000 7fe9d36f5640 1 Processor -- start 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.977+0000 7fe9d36f5640 1 -- start start 2026-02-22T23:24:41.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.977+0000 7fe9d36f5640 10 monclient: init 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient: _add_conns ranks=[2,4,1,0,3,5] 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe9cc06ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe9cc0c99a0 0x7fe9cc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient(hunting): picked mon.noname-b con 0x7fe9cc0c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fe9cc15a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9cc0c7e90 con 0x7fe9cc15a5e0 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9cc0c8170 con 0x7fe9cc06ff20 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe9cc159a40 con 0x7fe9cc0c99a0 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d36f5640 10 monclient(hunting): authenticate will time out at 1156.706312s 2026-02-22T23:24:41.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d0c69640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe9cc0c99a0 0x7fe9cc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d146a640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fe9cc06ff20 0x7fe9cc159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:34306/0 (socket says 10.20.193.6:34306) 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.978+0000 7fe9d146a640 1 -- 10.20.193.6:0/774532126 learned_addr learned my addr 10.20.193.6:0/774532126 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.979+0000 7fe9d0c69640 10 monclient(hunting): get_auth_request con 0x7fe9cc0c99a0 auth_method 0 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.979+0000 7fe9d0c69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.979+0000 7fe9d0c69640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.979+0000 7fe9d0c69640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.979+0000 7fe9d0c69640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.979+0000 7fe9d0c69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:41.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.979+0000 7fe9d0c69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9bbfff640 1 -- 10.20.193.6:0/774532126 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049816581 0 0) 0x7fe9cc0c8170 con 0x7fe9cc06ff20 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9bbfff640 10 monclient(hunting): _negotiate 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9bbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9bbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9bbfff640 10 monclient(hunting): my global_id is 4799 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9bbfff640 1 -- 10.20.193.6:0/774532126 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9b4003620 con 0x7fe9cc06ff20 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 10 monclient(hunting): handle_auth_done global_id 4717 payload 274 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 1 -- 10.20.193.6:0/774532126 >> v1:10.20.193.80:6789/0 conn(0x7fe9cc06ff20 legacy=0x7fe9cc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 1 -- 10.20.193.6:0/774532126 >> v1:10.20.193.40:6789/0 conn(0x7fe9cc15a5e0 legacy=0x7fe9cc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 10 monclient: _finish_hunting 0 2026-02-22T23:24:41.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 1 monclient: found mon.noname-b 2026-02-22T23:24:41.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:41.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:24:41.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.980+0000 7fe9d0c69640 1 -- 10.20.193.6:0/774532126 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc0c8450 con 0x7fe9cc0c99a0 2026-02-22T23:24:41.984 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.982+0000 7fe9d0c69640 1 --2- 10.20.193.6:0/774532126 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc0c99a0 0x7fe9cc070300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00098d0 tx=0x7fe9c002eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4f849fddddfddc server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 1 -- 10.20.193.6:0/774532126 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c003c070 con 0x7fe9cc0c99a0 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: dump: 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:41.986 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: _finish_auth 0 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: _check_auth_tickets 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 1 -- 10.20.193.6:0/774532126 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c002fba0 con 0x7fe9cc0c99a0 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 1 -- 10.20.193.6:0/774532126 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0037440 con 0x7fe9cc0c99a0 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 5 monclient: authenticate success, global_id 4717 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 4 set_mon_vals no callback set 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9bbfff640 10 monclient: dump: 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:41.987 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 10 monclient: shutdown 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 1 -- 10.20.193.6:0/774532126 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc0c99a0 msgr2=0x7fe9cc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/774532126 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc0c99a0 0x7fe9cc070300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00098d0 tx=0x7fe9c002eed0 comp rx=0 tx=0).stop 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 1 -- 10.20.193.6:0/774532126 shutdown_connections 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/774532126 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc0c99a0 0x7fe9cc070300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 1 -- 10.20.193.6:0/774532126 >> 10.20.193.6:0/774532126 conn(0x7fe9cc0cd780 msgr2=0x7fe9cc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 1 -- 10.20.193.6:0/774532126 shutdown_connections 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.984+0000 7fe9d36f5640 1 -- 10.20.193.6:0/774532126 wait complete. 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 10 monclient: build_initial_monmap 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 10 monclient: monmap: 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 1 librados: starting msgr at 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 1 librados: starting objecter 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 1 Processor -- start 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 1 -- start start 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 1 librados: setting wanted keys 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 1 librados: calling monclient init 2026-02-22T23:24:41.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.985+0000 7fe9d36f5640 10 monclient: init 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc06ff20 0x7fe9cc14bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): picked mon.noname-b con 0x7fe9cc06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe9cc0c99a0 0x7fe9cc156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): picked mon.noname-a con 0x7fe9cc0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe9cc15a5e0 0x7fe9cc156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): picked mon.noname-c con 0x7fe9cc15a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc150520 con 0x7fe9cc0c99a0 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc0c8a10 con 0x7fe9cc06ff20 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1506a0 con 0x7fe9cc15a5e0 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d36f5640 10 monclient(hunting): authenticate will time out at 1156.714276s 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc06ff20 0x7fe9cc14bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:41.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d0c69640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe9cc0c99a0 0x7fe9cc156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d1c6b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe9cc15a5e0 0x7fe9cc156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc06ff20 0x7fe9cc14bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:53046/0 (socket says 10.20.193.6:53046) 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 1 -- 10.20.193.6:0/3852080769 learned_addr learned my addr 10.20.193.6:0/3852080769 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 10 monclient(hunting): get_auth_request con 0x7fe9cc06ff20 auth_method 0 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d0c69640 10 monclient(hunting): get_auth_request con 0x7fe9cc0c99a0 auth_method 0 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d0c69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d0c69640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d0c69640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d1c6b640 10 monclient(hunting): get_auth_request con 0x7fe9cc15a5e0 auth_method 0 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d1c6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d1c6b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d1c6b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.986+0000 7fe9d146a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.987+0000 7fe9d146a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.987+0000 7fe9d0c69640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.987+0000 7fe9d0c69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.987+0000 7fe9d0c69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.987+0000 7fe9d1c6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.987+0000 7fe9d1c6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.987+0000 7fe9d1c6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 10 monclient(hunting): handle_auth_done global_id 4723 payload 824 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 1 -- 10.20.193.6:0/3852080769 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe9cc15a5e0 msgr2=0x7fe9cc156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe9cc15a5e0 0x7fe9cc156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 1 -- 10.20.193.6:0/3852080769 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe9cc0c99a0 msgr2=0x7fe9cc156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe9cc0c99a0 0x7fe9cc156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 10 monclient: _finish_hunting 0 2026-02-22T23:24:41.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 1 monclient: found mon.noname-b 2026-02-22T23:24:41.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:41.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:24:41.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.988+0000 7fe9d146a640 1 -- 10.20.193.6:0/3852080769 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc06ef20 con 0x7fe9cc06ff20 2026-02-22T23:24:41.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.989+0000 7fe9d146a640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc06ff20 0x7fe9cc14bec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c760 tx=0x7fe9bc00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:41.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.989+0000 7fe9b9ffb640 10 client.?.objecter ms_handle_connect 0x7fe9cc06ff20 2026-02-22T23:24:41.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.989+0000 7fe9b9ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 1 -- 10.20.193.6:0/3852080769 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc017070 con 0x7fe9cc06ff20 2026-02-22T23:24:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:24:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: dump: 2026-02-22T23:24:41.994 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: _finish_auth 0 2026-02-22T23:24:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 5 monclient: authenticate success, global_id 4723 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 10 monclient: _renew_subs 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc06f200 con 0x7fe9cc06ff20 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 10 client.4723.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 10 monclient: _renew_subs 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc155690 con 0x7fe9cc06ff20 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9d36f5640 1 librados: init done 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 1 -- 10.20.193.6:0/3852080769 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9bc005bb0 con 0x7fe9cc06ff20 2026-02-22T23:24:41.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 1 -- 10.20.193.6:0/3852080769 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc021060 con 0x7fe9cc06ff20 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9bb7fe640 4 set_mon_vals no callback set 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.990+0000 7fe9b9ffb640 10 monclient: dump: 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:41.997 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:41.998 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:41.998 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:41.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.991+0000 7fe9d36f5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:41.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.991+0000 7fe9d36f5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:41.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.991+0000 7fe9d36f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:24:41.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:41.991+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9cc1573f0 con 0x7fe9cc06ff20 2026-02-22T23:24:42.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.002+0000 7fe9b9ffb640 1 -- 10.20.193.6:0/3852080769 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe9bc024080 con 0x7fe9cc06ff20 2026-02-22T23:24:42.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.003+0000 7fe9b9ffb640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe99c078e00 0x7fe99c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:42.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.003+0000 7fe9d0c69640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe99c078e00 0x7fe99c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:42.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.003+0000 7fe9d0c69640 10 monclient: get_auth_request con 0x7fe99c078e00 auth_method 0 2026-02-22T23:24:42.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.003+0000 7fe9b9ffb640 1 -- 10.20.193.6:0/3852080769 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fe9bc05d5b0 con 0x7fe9cc06ff20 2026-02-22T23:24:42.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.003+0000 7fe9b9ffb640 10 client.4723.objecter ms_dispatch2 0x7fe9cc000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:42.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.003+0000 7fe9b9ffb640 3 client.4723.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:42.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.003+0000 7fe9b9ffb640 3 client.4723.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:42.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.004+0000 7fe9b9ffb640 20 client.4723.objecter dump_active .. 0 homeless 2026-02-22T23:24:42.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.004+0000 7fe9b9ffb640 10 client.4723.objecter ms_handle_connect 0x7fe99c078e00 2026-02-22T23:24:42.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.004+0000 7fe9d0c69640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe99c078e00 0x7fe99c07b2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0039000 tx=0x7fe9c0033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:42.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.104+0000 7fe9b9ffb640 1 -- 10.20.193.6:0/3852080769 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe9bc061570 con 0x7fe9cc06ff20 2026-02-22T23:24:42.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.104+0000 7fe9b9ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:42.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.104+0000 7fe9b9ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.204+0000 7fe9d36f5640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.204+0000 7fe9d36f5640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.204+0000 7fe9d36f5640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:24:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.204+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe9cc0c4a10 con 0x7fe9cc06ff20 2026-02-22T23:24:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9b9ffb640 1 -- 10.20.193.6:0/3852080769 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fe9bc065870 con 0x7fe9cc06ff20 2026-02-22T23:24:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9b9ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9b9ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:42.209 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:42.209 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:42.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 10 librados: watch_flush enter 2026-02-22T23:24:42.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 10 librados: watch_flush exit 2026-02-22T23:24:42.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 20 client.4723.objecter shutdown clearing up homeless session... 2026-02-22T23:24:42.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 10 client.4723.objecter successfully canceled tick 2026-02-22T23:24:42.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe99c078e00 msgr2=0x7fe99c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe99c078e00 0x7fe99c07b2c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0039000 tx=0x7fe9c0033000 comp rx=0 tx=0).stop 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 10 monclient: shutdown 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc06ff20 msgr2=0x7fe9cc14bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc06ff20 0x7fe9cc14bec0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c760 tx=0x7fe9bc00cc30 comp rx=0 tx=0).stop 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 shutdown_connections 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe99c078e00 0x7fe99c07b2c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe9cc15a5e0 0x7fe9cc156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe9cc0c99a0 0x7fe9cc156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:42.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 --2- 10.20.193.6:0/3852080769 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe9cc06ff20 0x7fe9cc14bec0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:42.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 >> 10.20.193.6:0/3852080769 conn(0x7fe9cc0cd780 msgr2=0x7fe9cc14e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:42.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 shutdown_connections 2026-02-22T23:24:42.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 -- 10.20.193.6:0/3852080769 wait complete. 2026-02-22T23:24:42.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:42.207+0000 7fe9d36f5640 1 librados: shutdown 2026-02-22T23:24:45.219 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:45.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.285+0000 7f85e3fff640 10 monclient: get_monmap_and_config 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.285+0000 7f85e3fff640 10 monclient: build_initial_monmap 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.285+0000 7f85e3fff640 10 monclient: monmap: 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:45.285936+0000 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:45.285936+0000 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:45.287 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:45.288 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:45.288 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:45.288 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:45.288 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:45.288 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:45.288 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:45.288 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:45.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.285+0000 7f85e3fff640 1 Processor -- start 2026-02-22T23:24:45.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.285+0000 7f85e3fff640 1 -- start start 2026-02-22T23:24:45.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.285+0000 7f85e3fff640 10 monclient: init 2026-02-22T23:24:45.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:45.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient: _add_conns ranks=[5,1,4,0,3,2] 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f85e4152510 0x7f85e415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f85e4152510 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f85e4159b40 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f85e415f8d0 0x7f85e415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient(hunting): picked mon.noname-b con 0x7f85e415f8d0 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85e415d820 con 0x7f85e4159b40 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f85e415d600 con 0x7f85e415f8d0 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f85e4159f90 con 0x7f85e4152510 2026-02-22T23:24:45.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.286+0000 7f85e3fff640 10 monclient(hunting): authenticate will time out at 1160.013627s 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e37fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f85e415f8d0 0x7f85e415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f85e4152510 0x7f85e415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f85e4152510 0x7f85e415eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:36310/0 (socket says 10.20.193.6:36310) 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 1 -- 10.20.193.6:0/4023745050 learned_addr learned my addr 10.20.193.6:0/4023745050 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 10 monclient(hunting): get_auth_request con 0x7f85e4152510 auth_method 0 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e37fe640 10 monclient(hunting): get_auth_request con 0x7f85e415f8d0 auth_method 0 2026-02-22T23:24:45.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.287+0000 7f85e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 10 monclient(hunting): handle_auth_done global_id 4805 payload 274 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 1 -- 10.20.193.6:0/4023745050 >> v2:10.20.193.40:3300/0 conn(0x7f85e415f8d0 msgr2=0x7f85e415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 1 --2- 10.20.193.6:0/4023745050 >> v2:10.20.193.40:3300/0 conn(0x7f85e415f8d0 0x7f85e415d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 1 -- 10.20.193.6:0/4023745050 >> v1:10.20.193.40:6789/0 conn(0x7f85e4159b40 legacy=0x7f85e415f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:45.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:24:45.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 1 monclient: found mon.noname-c 2026-02-22T23:24:45.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:45.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:45.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.288+0000 7f85e2ffd640 1 -- 10.20.193.6:0/4023745050 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e415db00 con 0x7f85e4152510 2026-02-22T23:24:45.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.290+0000 7f85e2ffd640 1 --2- 10.20.193.6:0/4023745050 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e4152510 0x7f85e415eba0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85e40699f0 tx=0x7f85d802eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4de4860674a567e0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.291+0000 7f85e1ffb640 1 -- 10.20.193.6:0/4023745050 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d803c070 con 0x7f85e4152510 2026-02-22T23:24:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:45.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: dump: 2026-02-22T23:24:45.302 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:45.302 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:45.302 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: _finish_auth 0 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:24:45.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 1 -- 10.20.193.6:0/4023745050 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d802fbe0 con 0x7f85e4152510 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 5 monclient: authenticate success, global_id 4805 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 1 -- 10.20.193.6:0/4023745050 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8037440 con 0x7f85e4152510 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 4 set_mon_vals no callback set 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e1ffb640 10 monclient: dump: 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:45.304 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 10 monclient: shutdown 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:45.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 -- 10.20.193.6:0/4023745050 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e4152510 msgr2=0x7f85e415eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 --2- 10.20.193.6:0/4023745050 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e4152510 0x7f85e415eba0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85e40699f0 tx=0x7f85d802eed0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 -- 10.20.193.6:0/4023745050 shutdown_connections 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 --2- 10.20.193.6:0/4023745050 >> v2:10.20.193.40:3300/0 conn(0x7f85e415f8d0 0x7f85e415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 --2- 10.20.193.6:0/4023745050 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e4152510 0x7f85e415eba0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 -- 10.20.193.6:0/4023745050 >> 10.20.193.6:0/4023745050 conn(0x7f85e40ce330 msgr2=0x7f85e40ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 -- 10.20.193.6:0/4023745050 shutdown_connections 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.292+0000 7f85e3fff640 1 -- 10.20.193.6:0/4023745050 wait complete. 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient: build_initial_monmap 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient: monmap: 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:45.306 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 librados: starting msgr at 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 librados: starting objecter 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 Processor -- start 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 -- start start 2026-02-22T23:24:45.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 librados: setting wanted keys 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 librados: calling monclient init 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient: init 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e4152510 0x7f85e415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): picked mon.noname-a con 0x7f85e4152510 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e4159b40 0x7f85e4152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): picked mon.noname-b con 0x7f85e4159b40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e415f8d0 0x7f85e4153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): picked mon.noname-c con 0x7f85e415f8d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f85e415e3a0 con 0x7f85e4152510 2026-02-22T23:24:45.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f85e415e0c0 con 0x7f85e4159b40 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f85e415e680 con 0x7f85e415f8d0 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e3fff640 10 monclient(hunting): authenticate will time out at 1160.020684s 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.293+0000 7f85e37fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e415f8d0 0x7f85e4153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e4152510 0x7f85e415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e4159b40 0x7f85e4152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e415f8d0 0x7f85e4153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:36312/0 (socket says 10.20.193.6:36312) 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 1 -- 10.20.193.6:0/165899152 learned_addr learned my addr 10.20.193.6:0/165899152 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 10 monclient(hunting): get_auth_request con 0x7f85e415f8d0 auth_method 0 2026-02-22T23:24:45.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 10 monclient(hunting): get_auth_request con 0x7f85e4152510 auth_method 0 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 10 monclient(hunting): get_auth_request con 0x7f85e4159b40 auth_method 0 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:45.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.294+0000 7f85e27fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 10 monclient(hunting): handle_auth_done global_id 4808 payload 824 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 1 -- 10.20.193.6:0/165899152 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e4159b40 msgr2=0x7f85e4152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e4159b40 0x7f85e4152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 1 -- 10.20.193.6:0/165899152 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e4152510 msgr2=0x7f85e415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e4152510 0x7f85e415b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 10 monclient: _finish_hunting 0 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 1 monclient: found mon.noname-c 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.295+0000 7f85e37fe640 1 -- 10.20.193.6:0/165899152 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e4153680 con 0x7f85e415f8d0 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.296+0000 7f85e37fe640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e415f8d0 0x7f85e4153040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f85d400efc0 tx=0x7f85d400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.296+0000 7f85c37fe640 10 client.?.objecter ms_handle_connect 0x7f85e415f8d0 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.296+0000 7f85c37fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85c37fe640 1 -- 10.20.193.6:0/165899152 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4018020 con 0x7f85e415f8d0 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85c37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85c37fe640 10 monclient: dump: 2026-02-22T23:24:45.312 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85c37fe640 10 monclient: _finish_auth 0 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85c37fe640 10 monclient: _check_auth_tickets 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85c37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:45.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 5 monclient: authenticate success, global_id 4808 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 10 monclient: _renew_subs 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e4153960 con 0x7f85e415f8d0 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 10 client.4808.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 10 monclient: _renew_subs 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e41f4700 con 0x7f85e415f8d0 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.297+0000 7f85e3fff640 1 librados: init done 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85c37fe640 1 -- 10.20.193.6:0/165899152 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d4014480 con 0x7f85e415f8d0 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85c37fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85c37fe640 1 -- 10.20.193.6:0/165899152 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4014830 con 0x7f85e415f8d0 2026-02-22T23:24:45.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85c37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85c37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85c37fe640 10 monclient: dump: 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:45.315 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:45.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85e17fa640 4 set_mon_vals no callback set 2026-02-22T23:24:45.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85e3fff640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:45.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85e3fff640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:45.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85e3fff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:45.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.298+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e41528f0 con 0x7f85e415f8d0 2026-02-22T23:24:45.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.399+0000 7f85c37fe640 1 -- 10.20.193.6:0/165899152 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f85d40149d0 con 0x7f85e415f8d0 2026-02-22T23:24:45.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.400+0000 7f85c37fe640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b0078e00 0x7f85b007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:45.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.400+0000 7f85e2ffd640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b0078e00 0x7f85b007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:45.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.400+0000 7f85e2ffd640 10 monclient: get_auth_request con 0x7f85b0078e00 auth_method 0 2026-02-22T23:24:45.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.400+0000 7f85e2ffd640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b0078e00 0x7f85b007b2c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f85e406cd60 tx=0x7f85d8039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:45.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.400+0000 7f85c37fe640 10 client.4808.objecter ms_handle_connect 0x7f85b0078e00 2026-02-22T23:24:45.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.401+0000 7f85c37fe640 1 -- 10.20.193.6:0/165899152 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f85d405f1b0 con 0x7f85e415f8d0 2026-02-22T23:24:45.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.401+0000 7f85c37fe640 10 client.4808.objecter ms_dispatch2 0x7f85e4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:45.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.401+0000 7f85c37fe640 3 client.4808.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:45.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.401+0000 7f85c37fe640 3 client.4808.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:45.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.401+0000 7f85c37fe640 20 client.4808.objecter dump_active .. 0 homeless 2026-02-22T23:24:45.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.407+0000 7f85c37fe640 1 -- 10.20.193.6:0/165899152 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f85d4022070 con 0x7f85e415f8d0 2026-02-22T23:24:45.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.407+0000 7f85c37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:45.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.407+0000 7f85c37fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:45.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.508+0000 7f85e3fff640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:45.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.508+0000 7f85e3fff640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:45.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.508+0000 7f85e3fff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:45.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.508+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f85e40cea90 con 0x7f85e415f8d0 2026-02-22T23:24:45.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.510+0000 7f85c37fe640 1 -- 10.20.193.6:0/165899152 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f85d4010040 con 0x7f85e415f8d0 2026-02-22T23:24:45.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.510+0000 7f85c37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:45.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.510+0000 7f85c37fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:45.513 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:45.513 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:45.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 10 librados: watch_flush enter 2026-02-22T23:24:45.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 10 librados: watch_flush exit 2026-02-22T23:24:45.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 20 client.4808.objecter shutdown clearing up homeless session... 2026-02-22T23:24:45.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 10 client.4808.objecter successfully canceled tick 2026-02-22T23:24:45.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b0078e00 msgr2=0x7f85b007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:45.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b0078e00 0x7f85b007b2c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f85e406cd60 tx=0x7f85d8039040 comp rx=0 tx=0).stop 2026-02-22T23:24:45.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 10 monclient: shutdown 2026-02-22T23:24:45.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:45.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e415f8d0 msgr2=0x7f85e4153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:45.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e415f8d0 0x7f85e4153040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f85d400efc0 tx=0x7f85d400c490 comp rx=0 tx=0).stop 2026-02-22T23:24:45.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 shutdown_connections 2026-02-22T23:24:45.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f85b0078e00 0x7f85b007b2c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f85e415f8d0 0x7f85e4153040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f85e4159b40 0x7f85e4152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 --2- 10.20.193.6:0/165899152 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f85e4152510 0x7f85e415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:45.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 >> 10.20.193.6:0/165899152 conn(0x7f85e40ce330 msgr2=0x7f85e415bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:45.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 shutdown_connections 2026-02-22T23:24:45.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 -- 10.20.193.6:0/165899152 wait complete. 2026-02-22T23:24:45.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:45.511+0000 7f85e3fff640 1 librados: shutdown 2026-02-22T23:24:48.523 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.592+0000 7ff95566f640 10 monclient: get_monmap_and_config 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.592+0000 7ff95566f640 10 monclient: build_initial_monmap 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.592+0000 7ff95566f640 10 monclient: monmap: 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:48.592698+0000 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:48.592698+0000 2026-02-22T23:24:48.594 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.592+0000 7ff95566f640 1 Processor -- start 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.592+0000 7ff95566f640 1 -- start start 2026-02-22T23:24:48.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.592+0000 7ff95566f640 10 monclient: init 2026-02-22T23:24:48.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:48.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient: _add_conns ranks=[2,5,1,0,3,4] 2026-02-22T23:24:48.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff95006ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:24:48.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff9500c99a0 0x7ff950070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:48.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient(hunting): picked mon.noname-c con 0x7ff9500c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff95015a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9500c7e90 con 0x7ff95015a5e0 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9500c8170 con 0x7ff95006ff20 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff950159a40 con 0x7ff9500c99a0 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff95566f640 10 monclient(hunting): authenticate will time out at 1163.320189s 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff94f7fe640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7ff95015a5e0 0x7ff95014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:48710/0 (socket says 10.20.193.6:48710) 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff94f7fe640 1 -- 10.20.193.6:0/1782941568 learned_addr learned my addr 10.20.193.6:0/1782941568 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.593+0000 7ff94e7fc640 1 --2- 10.20.193.6:0/1782941568 >> v2:10.20.193.80:3300/0 conn(0x7ff9500c99a0 0x7ff950070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:48.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.594+0000 7ff94e7fc640 10 monclient(hunting): get_auth_request con 0x7ff9500c99a0 auth_method 0 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.594+0000 7ff94e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.594+0000 7ff94e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.594+0000 7ff94e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.594+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.594+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.594+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94dffb640 1 -- 10.20.193.6:0/1782941568 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3083955140 0 0) 0x7ff9500c7e90 con 0x7ff95015a5e0 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94dffb640 10 monclient(hunting): _negotiate 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94dffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94dffb640 10 monclient(hunting): my global_id is 4732 2026-02-22T23:24:48.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94dffb640 1 -- 10.20.193.6:0/1782941568 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff93c003620 con 0x7ff95015a5e0 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_done global_id 4811 payload 274 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 1 -- 10.20.193.6:0/1782941568 >> v1:10.20.193.80:6789/0 conn(0x7ff95006ff20 legacy=0x7ff950159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 1 -- 10.20.193.6:0/1782941568 >> v1:10.20.193.40:6789/0 conn(0x7ff95015a5e0 legacy=0x7ff95014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 1 monclient: found mon.noname-c 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:48.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.595+0000 7ff94e7fc640 1 -- 10.20.193.6:0/1782941568 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9500c8450 con 0x7ff9500c99a0 2026-02-22T23:24:48.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.598+0000 7ff94e7fc640 1 --2- 10.20.193.6:0/1782941568 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff950070300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff9380098d0 tx=0x7ff93802eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d3ed1e2b9a9664 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 1 -- 10.20.193.6:0/1782941568 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93803c070 con 0x7ff9500c99a0 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: dump: 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:48.607 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: _finish_auth 0 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: _check_auth_tickets 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff95566f640 5 monclient: authenticate success, global_id 4811 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 1 -- 10.20.193.6:0/1782941568 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93802fba0 con 0x7ff9500c99a0 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff95566f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 1 -- 10.20.193.6:0/1782941568 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938037440 con 0x7ff9500c99a0 2026-02-22T23:24:48.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff94dffb640 10 monclient: dump: 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:48.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff95566f640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.600+0000 7ff95566f640 4 set_mon_vals no callback set 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 10 monclient: shutdown 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 -- 10.20.193.6:0/1782941568 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 msgr2=0x7ff950070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 --2- 10.20.193.6:0/1782941568 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff950070300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff9380098d0 tx=0x7ff93802eed0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 -- 10.20.193.6:0/1782941568 shutdown_connections 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 --2- 10.20.193.6:0/1782941568 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff950070300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 -- 10.20.193.6:0/1782941568 >> 10.20.193.6:0/1782941568 conn(0x7ff9500cd780 msgr2=0x7ff9500cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 -- 10.20.193.6:0/1782941568 shutdown_connections 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 -- 10.20.193.6:0/1782941568 wait complete. 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 10 monclient: build_initial_monmap 2026-02-22T23:24:48.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 10 monclient: monmap: 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 librados: starting msgr at 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 librados: starting objecter 2026-02-22T23:24:48.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 Processor -- start 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 -- start start 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 librados: setting wanted keys 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 1 librados: calling monclient init 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.601+0000 7ff95566f640 10 monclient: init 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff95006ff20 0x7ff9501f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): picked mon.noname-b con 0x7ff95006ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff9501f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): picked mon.noname-c con 0x7ff9500c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff95015a5e0 0x7ff95006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:48.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): picked mon.noname-a con 0x7ff95015a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff950150520 con 0x7ff95015a5e0 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff9500c8a10 con 0x7ff95006ff20 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9501506a0 con 0x7ff9500c99a0 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff95566f640 10 monclient(hunting): authenticate will time out at 1163.329309s 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94e7fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff9501f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94effd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff95006ff20 0x7ff9501f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:48.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94f7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff95015a5e0 0x7ff95006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:48.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94e7fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff9501f7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:46450/0 (socket says 10.20.193.6:46450) 2026-02-22T23:24:48.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94e7fc640 1 -- 10.20.193.6:0/928458339 learned_addr learned my addr 10.20.193.6:0/928458339 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:48.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94e7fc640 10 monclient(hunting): get_auth_request con 0x7ff9500c99a0 auth_method 0 2026-02-22T23:24:48.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:48.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:48.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:48.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94effd640 10 monclient(hunting): get_auth_request con 0x7ff95006ff20 auth_method 0 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94f7fe640 10 monclient(hunting): get_auth_request con 0x7ff95015a5e0 auth_method 0 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.602+0000 7ff94f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:48.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.603+0000 7ff94f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 10 monclient(hunting): handle_auth_done global_id 4817 payload 824 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 1 -- 10.20.193.6:0/928458339 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff95006ff20 msgr2=0x7ff9501f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff95006ff20 0x7ff9501f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 1 -- 10.20.193.6:0/928458339 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff95015a5e0 msgr2=0x7ff95006e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff95015a5e0 0x7ff95006e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 1 monclient: found mon.noname-c 2026-02-22T23:24:48.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:48.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:48.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.604+0000 7ff94e7fc640 1 -- 10.20.193.6:0/928458339 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff950155a00 con 0x7ff9500c99a0 2026-02-22T23:24:48.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.605+0000 7ff94e7fc640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff9501f7710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff93803a040 tx=0x7ff9380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:48.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.605+0000 7ff92f7fe640 10 client.?.objecter ms_handle_connect 0x7ff9500c99a0 2026-02-22T23:24:48.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.605+0000 7ff92f7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 1 -- 10.20.193.6:0/928458339 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93803c040 con 0x7ff9500c99a0 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: dump: 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.619 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: _finish_auth 0 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 1 -- 10.20.193.6:0/928458339 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff938004230 con 0x7ff9500c99a0 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:48.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 5 monclient: authenticate success, global_id 4817 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff94d7fa640 4 set_mon_vals no callback set 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 monclient: _renew_subs 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 1 -- 10.20.193.6:0/928458339 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938004620 con 0x7ff9500c99a0 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff950155ce0 con 0x7ff9500c99a0 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff92f7fe640 10 monclient: dump: 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:48.621 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 client.4817.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 monclient: _renew_subs 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501562a0 con 0x7ff9500c99a0 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 1 librados: init done 2026-02-22T23:24:48.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:48.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:48.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:48.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.607+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9501573f0 con 0x7ff9500c99a0 2026-02-22T23:24:48.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.618+0000 7ff92f7fe640 1 -- 10.20.193.6:0/928458339 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff938003710 con 0x7ff9500c99a0 2026-02-22T23:24:48.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.619+0000 7ff92f7fe640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff91c078e00 0x7ff91c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:48.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.619+0000 7ff94effd640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff91c078e00 0x7ff91c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:48.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.619+0000 7ff94effd640 10 monclient: get_auth_request con 0x7ff91c078e00 auth_method 0 2026-02-22T23:24:48.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.620+0000 7ff94effd640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff91c078e00 0x7ff91c07b2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff944002770 tx=0x7ff944007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:48.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.620+0000 7ff92f7fe640 10 client.4817.objecter ms_handle_connect 0x7ff91c078e00 2026-02-22T23:24:48.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.621+0000 7ff92f7fe640 1 -- 10.20.193.6:0/928458339 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7ff93802fa60 con 0x7ff9500c99a0 2026-02-22T23:24:48.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.621+0000 7ff92f7fe640 10 client.4817.objecter ms_dispatch2 0x7ff950000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:48.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.621+0000 7ff92f7fe640 3 client.4817.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:48.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.621+0000 7ff92f7fe640 3 client.4817.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:48.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.621+0000 7ff92f7fe640 20 client.4817.objecter dump_active .. 0 homeless 2026-02-22T23:24:48.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.721+0000 7ff92f7fe640 1 -- 10.20.193.6:0/928458339 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff938050070 con 0x7ff9500c99a0 2026-02-22T23:24:48.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.721+0000 7ff92f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:48.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.721+0000 7ff92f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:48.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.821+0000 7ff95566f640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:48.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.821+0000 7ff95566f640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:48.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.821+0000 7ff95566f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:48.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.821+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff9501585d0 con 0x7ff9500c99a0 2026-02-22T23:24:48.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.824+0000 7ff92f7fe640 1 -- 10.20.193.6:0/928458339 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff938046020 con 0x7ff9500c99a0 2026-02-22T23:24:48.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.824+0000 7ff92f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:48.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.824+0000 7ff92f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:48.826 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:48.826 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:48.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 10 librados: watch_flush enter 2026-02-22T23:24:48.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 10 librados: watch_flush exit 2026-02-22T23:24:48.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 20 client.4817.objecter shutdown clearing up homeless session... 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 10 client.4817.objecter successfully canceled tick 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff91c078e00 msgr2=0x7ff91c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff91c078e00 0x7ff91c07b2c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff944002770 tx=0x7ff944007ef0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 10 monclient: shutdown 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 msgr2=0x7ff9501f7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff9501f7710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff93803a040 tx=0x7ff9380047c0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 shutdown_connections 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff91c078e00 0x7ff91c07b2c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff95015a5e0 0x7ff95006e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff9500c99a0 0x7ff9501f7710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 --2- 10.20.193.6:0/928458339 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff95006ff20 0x7ff9501f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:48.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 >> 10.20.193.6:0/928458339 conn(0x7ff9500cd780 msgr2=0x7ff95014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:48.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 shutdown_connections 2026-02-22T23:24:48.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 -- 10.20.193.6:0/928458339 wait complete. 2026-02-22T23:24:48.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:48.825+0000 7ff95566f640 1 librados: shutdown 2026-02-22T23:24:51.836 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:51.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.904+0000 7feac3085640 10 monclient: get_monmap_and_config 2026-02-22T23:24:51.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.904+0000 7feac3085640 10 monclient: build_initial_monmap 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.904+0000 7feac3085640 10 monclient: monmap: 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:51.904750+0000 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:51.904750+0000 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:51.908 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.904+0000 7feac3085640 1 Processor -- start 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.904+0000 7feac3085640 1 -- start start 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.904+0000 7feac3085640 10 monclient: init 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient: _add_conns ranks=[5,3,1,4,2,0] 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7feabc06ff20 0x7feabc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient(hunting): picked mon.noname-c con 0x7feabc06ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7feabc0c99a0 0x7feabc159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:51.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient(hunting): picked mon.noname-a con 0x7feabc0c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7feabc15a410 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feabc15a860 con 0x7feabc15a410 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7feabc159a40 con 0x7feabc0c99a0 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7feabc0c7e90 con 0x7feabc06ff20 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac3085640 10 monclient(hunting): authenticate will time out at 1166.632140s 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feabbfff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7feabc0c99a0 0x7feabc159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:51.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac0dfa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7feabc06ff20 0x7feabc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac15fb640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7feabc15a410 0x7feabc14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:48718/0 (socket says 10.20.193.6:48718) 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac15fb640 1 -- 10.20.193.6:0/2450410366 learned_addr learned my addr 10.20.193.6:0/2450410366 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feabbfff640 10 monclient(hunting): get_auth_request con 0x7feabc0c99a0 auth_method 0 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feabbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feabbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feabbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac0dfa640 10 monclient(hunting): get_auth_request con 0x7feabc06ff20 auth_method 0 2026-02-22T23:24:51.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac0dfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac0dfa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.905+0000 7feac0dfa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.906+0000 7feabbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.906+0000 7feabbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.906+0000 7feabbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.906+0000 7feac0dfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.906+0000 7feac0dfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:51.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.906+0000 7feac0dfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:51.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 10 monclient(hunting): handle_auth_done global_id 4710 payload 274 2026-02-22T23:24:51.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 1 -- 10.20.193.6:0/2450410366 >> v2:10.20.193.80:3300/0 conn(0x7feabc06ff20 msgr2=0x7feabc070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:51.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 1 --2- 10.20.193.6:0/2450410366 >> v2:10.20.193.80:3300/0 conn(0x7feabc06ff20 0x7feabc070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:51.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 1 -- 10.20.193.6:0/2450410366 >> v1:10.20.193.40:6789/0 conn(0x7feabc15a410 legacy=0x7feabc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:51.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 10 monclient: _finish_hunting 0 2026-02-22T23:24:51.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 1 monclient: found mon.noname-a 2026-02-22T23:24:51.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:51.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 1 -- 10.20.193.6:0/2450410366 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc0c8170 con 0x7feabc0c99a0 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.907+0000 7feabbfff640 1 --2- 10.20.193.6:0/2450410366 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc0c99a0 0x7feabc159ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feaac0098a0 tx=0x7feaac02ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31105c6b8e5f6c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 1 -- 10.20.193.6:0/2450410366 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac03c070 con 0x7feabc0c99a0 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: dump: 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:51.925 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:51.926 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: _finish_auth 0 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feac3085640 5 monclient: authenticate success, global_id 4710 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 1 -- 10.20.193.6:0/2450410366 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac02fae0 con 0x7feabc0c99a0 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feac3085640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:51.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feac3085640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 1 -- 10.20.193.6:0/2450410366 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac037440 con 0x7feabc0c99a0 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feac3085640 4 set_mon_vals no callback set 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.909+0000 7feabb7fe640 10 monclient: dump: 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:51.928 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:51.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 10 monclient: shutdown 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 -- 10.20.193.6:0/2450410366 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc0c99a0 msgr2=0x7feabc159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 --2- 10.20.193.6:0/2450410366 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc0c99a0 0x7feabc159ed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feaac0098a0 tx=0x7feaac02ee10 comp rx=0 tx=0).stop 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 -- 10.20.193.6:0/2450410366 shutdown_connections 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 --2- 10.20.193.6:0/2450410366 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc0c99a0 0x7feabc159ed0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 --2- 10.20.193.6:0/2450410366 >> v2:10.20.193.80:3300/0 conn(0x7feabc06ff20 0x7feabc070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 -- 10.20.193.6:0/2450410366 >> 10.20.193.6:0/2450410366 conn(0x7feabc0cd780 msgr2=0x7feabc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:51.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 -- 10.20.193.6:0/2450410366 shutdown_connections 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 -- 10.20.193.6:0/2450410366 wait complete. 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 10 monclient: build_initial_monmap 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 10 monclient: monmap: 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:51.931 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 librados: starting msgr at 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 librados: starting objecter 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 Processor -- start 2026-02-22T23:24:51.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 -- start start 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 librados: setting wanted keys 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 1 librados: calling monclient init 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.910+0000 7feac3085640 10 monclient: init 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc06ff20 0x7feabc1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): picked mon.noname-a con 0x7feabc06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:51.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feabc0c99a0 0x7feabc1f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): picked mon.noname-b con 0x7feabc0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feabc15a410 0x7feabc1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): picked mon.noname-c con 0x7feabc15a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7feabc0c8a10 con 0x7feabc06ff20 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7feabc0c8730 con 0x7feabc0c99a0 2026-02-22T23:24:51.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7feabc150520 con 0x7feabc15a410 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac3085640 10 monclient(hunting): authenticate will time out at 1166.638365s 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc06ff20 0x7feabc1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc06ff20 0x7feabc1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39166/0 (socket says 10.20.193.6:39166) 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 1 -- 10.20.193.6:0/1240917758 learned_addr learned my addr 10.20.193.6:0/1240917758 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 10 monclient(hunting): get_auth_request con 0x7feabc06ff20 auth_method 0 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feabbfff640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feabc0c99a0 0x7feabc1f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:51.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac15fb640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feabc15a410 0x7feabc1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.911+0000 7feac0dfa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feac15fb640 10 monclient(hunting): get_auth_request con 0x7feabc15a410 auth_method 0 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feac15fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feac15fb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feac15fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feabbfff640 10 monclient(hunting): get_auth_request con 0x7feabc0c99a0 auth_method 0 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feabbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feabbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feabbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feac15fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:51.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feac15fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feac15fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feabbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feabbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.912+0000 7feabbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 10 monclient(hunting): handle_auth_done global_id 4713 payload 824 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 1 -- 10.20.193.6:0/1240917758 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feabc15a410 msgr2=0x7feabc1f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feabc15a410 0x7feabc1f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 1 -- 10.20.193.6:0/1240917758 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feabc0c99a0 msgr2=0x7feabc1f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feabc0c99a0 0x7feabc1f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 10 monclient: _finish_hunting 0 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 1 monclient: found mon.noname-a 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:51.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.913+0000 7feac0dfa640 1 -- 10.20.193.6:0/1240917758 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc156010 con 0x7feabc06ff20 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.914+0000 7feac0dfa640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc06ff20 0x7feabc1559b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feaa800ebe0 tx=0x7feaa800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.914+0000 7feab97fa640 10 client.?.objecter ms_handle_connect 0x7feabc06ff20 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.914+0000 7feab97fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 1 -- 10.20.193.6:0/1240917758 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8017070 con 0x7feabc06ff20 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: dump: 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.938 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: _finish_auth 0 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: _check_auth_tickets 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 1 -- 10.20.193.6:0/1240917758 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa800cc40 con 0x7feabc06ff20 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 5 monclient: authenticate success, global_id 4713 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 monclient: _renew_subs 2026-02-22T23:24:51.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc1562f0 con 0x7feabc06ff20 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 1 -- 10.20.193.6:0/1240917758 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8013470 con 0x7feabc06ff20 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feab97fa640 10 monclient: dump: 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:51.940 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feabaffd640 4 set_mon_vals no callback set 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 client.4713.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 monclient: _renew_subs 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc1568b0 con 0x7feabc06ff20 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 1 librados: init done 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:51.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:51.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:51.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.916+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc070300 con 0x7feabc06ff20 2026-02-22T23:24:51.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.927+0000 7feab97fa640 1 -- 10.20.193.6:0/1240917758 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7feaa8013610 con 0x7feabc06ff20 2026-02-22T23:24:51.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.928+0000 7feab97fa640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fea94078e30 0x7fea9407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.928+0000 7feabbfff640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fea94078e30 0x7fea9407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.928+0000 7feabbfff640 10 monclient: get_auth_request con 0x7fea94078e30 auth_method 0 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.929+0000 7feabbfff640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fea94078e30 0x7fea9407b2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feaac039000 tx=0x7feaac033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.929+0000 7feab97fa640 10 client.4713.objecter ms_handle_connect 0x7fea94078e30 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.929+0000 7feab97fa640 1 -- 10.20.193.6:0/1240917758 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7feaa805e540 con 0x7feabc06ff20 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.929+0000 7feab97fa640 10 client.4713.objecter ms_dispatch2 0x7feabc000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.929+0000 7feab97fa640 3 client.4713.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.929+0000 7feab97fa640 3 client.4713.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:51.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:51.929+0000 7feab97fa640 20 client.4713.objecter dump_active .. 0 homeless 2026-02-22T23:24:52.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.028+0000 7feab97fa640 1 -- 10.20.193.6:0/1240917758 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7feaa8062500 con 0x7feabc06ff20 2026-02-22T23:24:52.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.028+0000 7feab97fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:52.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.028+0000 7feab97fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:52.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.129+0000 7feac3085640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:52.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.129+0000 7feac3085640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:52.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.129+0000 7feac3085640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:52.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.129+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7feabc150190 con 0x7feabc06ff20 2026-02-22T23:24:52.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.131+0000 7feab97fa640 1 -- 10.20.193.6:0/1240917758 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7feaa8066800 con 0x7feabc06ff20 2026-02-22T23:24:52.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.131+0000 7feab97fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:52.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.131+0000 7feab97fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:52.134 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:52.134 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:52.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 10 librados: watch_flush enter 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 10 librados: watch_flush exit 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 20 client.4713.objecter shutdown clearing up homeless session... 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 10 client.4713.objecter successfully canceled tick 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fea94078e30 msgr2=0x7fea9407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fea94078e30 0x7fea9407b2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feaac039000 tx=0x7feaac033000 comp rx=0 tx=0).stop 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 10 monclient: shutdown 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc06ff20 msgr2=0x7feabc1559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc06ff20 0x7feabc1559b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feaa800ebe0 tx=0x7feaa800c520 comp rx=0 tx=0).stop 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 shutdown_connections 2026-02-22T23:24:52.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fea94078e30 0x7fea9407b2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:52.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7feabc15a410 0x7feabc1f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:52.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7feabc0c99a0 0x7feabc1f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:52.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 --2- 10.20.193.6:0/1240917758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7feabc06ff20 0x7feabc1559b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:52.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 >> 10.20.193.6:0/1240917758 conn(0x7feabc0cd780 msgr2=0x7feabc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:52.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 shutdown_connections 2026-02-22T23:24:52.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 -- 10.20.193.6:0/1240917758 wait complete. 2026-02-22T23:24:52.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:52.132+0000 7feac3085640 1 librados: shutdown 2026-02-22T23:24:52.985 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:37.235 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.012 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:37.262 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.013 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:37.259 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.014 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:37.261 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.021 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:37.271 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.022 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:37.269 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.037 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:37.248 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.038 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:37.249 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:53.045 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:37.257 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:24:55.143 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.211+0000 7fef141dc640 10 monclient: get_monmap_and_config 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.211+0000 7fef141dc640 10 monclient: build_initial_monmap 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.211+0000 7fef141dc640 10 monclient: monmap: 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:55.212417+0000 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:55.212417+0000 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 1 Processor -- start 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 1 -- start start 2026-02-22T23:24:55.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient: init 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient: _add_conns ranks=[1,5,3,4,2,0] 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fef0c06ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fef0c0c99a0 0x7fef0c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient(hunting): picked mon.noname-c con 0x7fef0c0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fef0c15a5e0 0x7fef0c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient(hunting): picked mon.noname-a con 0x7fef0c15a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0c0c7e90 con 0x7fef0c06ff20 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c159a40 con 0x7fef0c15a5e0 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:55.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c0c8170 con 0x7fef0c0c99a0 2026-02-22T23:24:55.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:55.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.212+0000 7fef141dc640 10 monclient(hunting): authenticate will time out at 1169.939903s 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fef0c15a5e0 0x7fef0c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fef0c0c99a0 0x7fef0c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11f51640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fef0c06ff20 0x7fef0c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:48726/0 (socket says 10.20.193.6:48726) 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11f51640 1 -- 10.20.193.6:0/3681452830 learned_addr learned my addr 10.20.193.6:0/3681452830 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 10 monclient(hunting): get_auth_request con 0x7fef0c15a5e0 auth_method 0 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 10 monclient(hunting): get_auth_request con 0x7fef0c0c99a0 auth_method 0 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:55.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef12752640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.213+0000 7fef11750640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef10f4f640 1 -- 10.20.193.6:0/3681452830 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797483366 0 0) 0x7fef0c0c7e90 con 0x7fef0c06ff20 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef10f4f640 10 monclient(hunting): _negotiate 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef10f4f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef10f4f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef10f4f640 10 monclient(hunting): my global_id is 4744 2026-02-22T23:24:55.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef10f4f640 1 -- 10.20.193.6:0/3681452830 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feef4003620 con 0x7fef0c06ff20 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 10 monclient(hunting): handle_auth_done global_id 4716 payload 274 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 1 -- 10.20.193.6:0/3681452830 >> v2:10.20.193.80:3300/0 conn(0x7fef0c0c99a0 msgr2=0x7fef0c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 1 --2- 10.20.193.6:0/3681452830 >> v2:10.20.193.80:3300/0 conn(0x7fef0c0c99a0 0x7fef0c070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 1 -- 10.20.193.6:0/3681452830 >> v1:10.20.193.40:6789/0 conn(0x7fef0c06ff20 legacy=0x7fef0c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 10 monclient: _finish_hunting 0 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 1 monclient: found mon.noname-a 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:55.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.215+0000 7fef12752640 1 -- 10.20.193.6:0/3681452830 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c0c8450 con 0x7fef0c15a5e0 2026-02-22T23:24:55.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.216+0000 7fef12752640 1 --2- 10.20.193.6:0/3681452830 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c15a5e0 0x7fef0c14fe10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef08009d90 tx=0x7fef0802f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e6a70fa42cfb0a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:55.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 1 -- 10.20.193.6:0/3681452830 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0803c070 con 0x7fef0c15a5e0 2026-02-22T23:24:55.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:55.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:55.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: dump: 2026-02-22T23:24:55.225 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:55.225 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: _finish_auth 0 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: _check_auth_tickets 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:55.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 1 -- 10.20.193.6:0/3681452830 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef08037440 con 0x7fef0c15a5e0 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 1 -- 10.20.193.6:0/3681452830 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08037810 con 0x7fef0c15a5e0 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 5 monclient: authenticate success, global_id 4716 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef10f4f640 10 monclient: dump: 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:55.227 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 4 set_mon_vals no callback set 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 10 monclient: shutdown 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 -- 10.20.193.6:0/3681452830 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c15a5e0 msgr2=0x7fef0c14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 --2- 10.20.193.6:0/3681452830 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c15a5e0 0x7fef0c14fe10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef08009d90 tx=0x7fef0802f1e0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 -- 10.20.193.6:0/3681452830 shutdown_connections 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 --2- 10.20.193.6:0/3681452830 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c15a5e0 0x7fef0c14fe10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 --2- 10.20.193.6:0/3681452830 >> v2:10.20.193.80:3300/0 conn(0x7fef0c0c99a0 0x7fef0c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 -- 10.20.193.6:0/3681452830 >> 10.20.193.6:0/3681452830 conn(0x7fef0c0cd780 msgr2=0x7fef0c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 -- 10.20.193.6:0/3681452830 shutdown_connections 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.218+0000 7fef141dc640 1 -- 10.20.193.6:0/3681452830 wait complete. 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient: build_initial_monmap 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient: monmap: 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:55.229 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 librados: starting msgr at 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 librados: starting objecter 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 Processor -- start 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 -- start start 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 librados: setting wanted keys 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 librados: calling monclient init 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient: init 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef0c06ff20 0x7fef0c0c0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:55.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): picked mon.noname-c con 0x7fef0c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c0c99a0 0x7fef0c1f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): picked mon.noname-a con 0x7fef0c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef0c15a5e0 0x7fef0c072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): picked mon.noname-b con 0x7fef0c15a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c150350 con 0x7fef0c0c99a0 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c0c8a10 con 0x7fef0c15a5e0 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1504d0 con 0x7fef0c06ff20 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.219+0000 7fef141dc640 10 monclient(hunting): authenticate will time out at 1169.946769s 2026-02-22T23:24:55.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c0c99a0 0x7fef0c1f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef0c06ff20 0x7fef0c0c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef0c15a5e0 0x7fef0c072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c0c99a0 0x7fef0c1f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:39180/0 (socket says 10.20.193.6:39180) 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 1 -- 10.20.193.6:0/2800341317 learned_addr learned my addr 10.20.193.6:0/2800341317 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 10 monclient(hunting): get_auth_request con 0x7fef0c0c99a0 auth_method 0 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 10 monclient(hunting): get_auth_request con 0x7fef0c06ff20 auth_method 0 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:55.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 10 monclient(hunting): get_auth_request con 0x7fef0c15a5e0 auth_method 0 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11750640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:55.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef11f51640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.220+0000 7fef12752640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 10 monclient(hunting): handle_auth_done global_id 4829 payload 824 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 1 -- 10.20.193.6:0/2800341317 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef0c15a5e0 msgr2=0x7fef0c072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef0c15a5e0 0x7fef0c072eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 1 -- 10.20.193.6:0/2800341317 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c0c99a0 msgr2=0x7fef0c1f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c0c99a0 0x7fef0c1f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 10 monclient: _finish_hunting 0 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 1 monclient: found mon.noname-c 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:24:55.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.221+0000 7fef11f51640 1 -- 10.20.193.6:0/2800341317 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c0befb0 con 0x7fef0c06ff20 2026-02-22T23:24:55.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.223+0000 7fef11f51640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef0c06ff20 0x7fef0c0c0050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c530 tx=0x7feefc00ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:55.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.223+0000 7feefa7fc640 10 client.?.objecter ms_handle_connect 0x7fef0c06ff20 2026-02-22T23:24:55.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.223+0000 7feefa7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:55.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 1 -- 10.20.193.6:0/2800341317 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc017070 con 0x7fef0c06ff20 2026-02-22T23:24:55.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 10 monclient: dump: 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:55.237 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 10 monclient: _finish_auth 0 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 1 -- 10.20.193.6:0/2800341317 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc005de0 con 0x7fef0c06ff20 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7fef141dc640 5 monclient: authenticate success, global_id 4829 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7fef141dc640 10 monclient: _renew_subs 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7fef141dc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c0bf290 con 0x7fef0c06ff20 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.224+0000 7feefa7fc640 1 -- 10.20.193.6:0/2800341317 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc007270 con 0x7fef0c06ff20 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7feefa7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7feefbfff640 4 set_mon_vals no callback set 2026-02-22T23:24:55.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7feefa7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7feefa7fc640 10 monclient: dump: 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:55.239 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 10 client.4829.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 10 monclient: _renew_subs 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c0bf850 con 0x7fef0c06ff20 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 1 librados: init done 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:55.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.225+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c150860 con 0x7fef0c06ff20 2026-02-22T23:24:55.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.236+0000 7feefa7fc640 1 -- 10.20.193.6:0/2800341317 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7feefc007410 con 0x7fef0c06ff20 2026-02-22T23:24:55.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.237+0000 7feefa7fc640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feedc078e00 0x7feedc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.237+0000 7fef11750640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feedc078e00 0x7feedc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.237+0000 7fef11750640 10 monclient: get_auth_request con 0x7feedc078e00 auth_method 0 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.237+0000 7fef11750640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feedc078e00 0x7feedc07b2c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef0c14e6e0 tx=0x7fef00009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.237+0000 7feefa7fc640 10 client.4829.objecter ms_handle_connect 0x7feedc078e00 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.239+0000 7feefa7fc640 1 -- 10.20.193.6:0/2800341317 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7feefc007730 con 0x7fef0c06ff20 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.239+0000 7feefa7fc640 10 client.4829.objecter ms_dispatch2 0x7fef0c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.239+0000 7feefa7fc640 3 client.4829.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.239+0000 7feefa7fc640 3 client.4829.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:55.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.239+0000 7feefa7fc640 20 client.4829.objecter dump_active .. 0 homeless 2026-02-22T23:24:55.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.337+0000 7feefa7fc640 1 -- 10.20.193.6:0/2800341317 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7feefc061550 con 0x7fef0c06ff20 2026-02-22T23:24:55.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.337+0000 7feefa7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:55.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.337+0000 7feefa7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:55.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.435+0000 7fef141dc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:55.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.435+0000 7fef141dc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:55.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.435+0000 7fef141dc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:24:55.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.435+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fef0c0c4bc0 con 0x7fef0c06ff20 2026-02-22T23:24:55.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.439+0000 7feefa7fc640 1 -- 10.20.193.6:0/2800341317 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7feefc065850 con 0x7fef0c06ff20 2026-02-22T23:24:55.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.439+0000 7feefa7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:55.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.439+0000 7feefa7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:55.441 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:55.441 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:55.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 10 librados: watch_flush enter 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 10 librados: watch_flush exit 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 20 client.4829.objecter shutdown clearing up homeless session... 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 10 client.4829.objecter successfully canceled tick 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feedc078e00 msgr2=0x7feedc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feedc078e00 0x7feedc07b2c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef0c14e6e0 tx=0x7fef00009290 comp rx=0 tx=0).stop 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 10 monclient: shutdown 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef0c06ff20 msgr2=0x7fef0c0c0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef0c06ff20 0x7fef0c0c0050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c530 tx=0x7feefc00ca00 comp rx=0 tx=0).stop 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 shutdown_connections 2026-02-22T23:24:55.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feedc078e00 0x7feedc07b2c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef0c15a5e0 0x7fef0c072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef0c0c99a0 0x7fef0c1f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 --2- 10.20.193.6:0/2800341317 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef0c06ff20 0x7fef0c0c0050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:55.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 >> 10.20.193.6:0/2800341317 conn(0x7fef0c0cd780 msgr2=0x7fef0c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:55.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 shutdown_connections 2026-02-22T23:24:55.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 -- 10.20.193.6:0/2800341317 wait complete. 2026-02-22T23:24:55.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:55.440+0000 7fef141dc640 1 librados: shutdown 2026-02-22T23:24:58.451 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.518+0000 7f3c5ae7c640 10 monclient: get_monmap_and_config 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.518+0000 7f3c5ae7c640 10 monclient: build_initial_monmap 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.518+0000 7f3c5ae7c640 10 monclient: monmap: 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:24:58.519270+0000 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:24:58.519270+0000 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:24:58.520 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 1 Processor -- start 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 1 -- start start 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient: init 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient: _add_conns ranks=[4,1,0,3,5,2] 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f3c5406ff20 0x7f3c54070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient(hunting): picked mon.noname-b con 0x7f3c5406ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3c540c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c5415a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c540c7e90 con 0x7f3c5415a5e0 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c540c8170 con 0x7f3c540c99a0 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3c54159a40 con 0x7f3c5406ff20 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.519+0000 7f3c5ae7c640 10 monclient(hunting): authenticate will time out at 1173.246754s 2026-02-22T23:24:58.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.520+0000 7f3c593f2640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f3c5415a5e0 0x7f3c5414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:60522/0 (socket says 10.20.193.6:60522) 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.520+0000 7f3c593f2640 1 -- 10.20.193.6:0/2870163606 learned_addr learned my addr 10.20.193.6:0/2870163606 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.520+0000 7f3c58bf1640 1 --2- 10.20.193.6:0/2870163606 >> v2:10.20.193.40:3300/0 conn(0x7f3c5406ff20 0x7f3c54070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.520+0000 7f3c58bf1640 10 monclient(hunting): get_auth_request con 0x7f3c5406ff20 auth_method 0 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.520+0000 7f3c58bf1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.520+0000 7f3c58bf1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.520+0000 7f3c58bf1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c4b7fe640 1 -- 10.20.193.6:0/2870163606 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3846519250 0 0) 0x7f3c540c7e90 con 0x7f3c5415a5e0 2026-02-22T23:24:58.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c4b7fe640 10 monclient(hunting): _negotiate 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c4b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c4b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c4b7fe640 10 monclient(hunting): my global_id is 4722 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.521+0000 7f3c4b7fe640 1 -- 10.20.193.6:0/2870163606 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c40003620 con 0x7f3c5415a5e0 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_done global_id 4750 payload 274 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 1 -- 10.20.193.6:0/2870163606 >> v1:10.20.193.40:6789/0 conn(0x7f3c540c99a0 legacy=0x7f3c54159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 1 -- 10.20.193.6:0/2870163606 >> v1:10.20.193.6:6789/0 conn(0x7f3c5415a5e0 legacy=0x7f3c5414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 10 monclient: _finish_hunting 0 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 1 monclient: found mon.noname-b 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:24:58.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.522+0000 7f3c58bf1640 1 -- 10.20.193.6:0/2870163606 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c540c8450 con 0x7f3c5406ff20 2026-02-22T23:24:58.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.524+0000 7f3c58bf1640 1 --2- 10.20.193.6:0/2870163606 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5406ff20 0x7f3c54070300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c440098d0 tx=0x7f3c4402eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458af13b339c39fc server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:58.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 1 -- 10.20.193.6:0/2870163606 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4403c070 con 0x7f3c5406ff20 2026-02-22T23:24:58.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: dump: 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 1 -- 10.20.193.6:0/2870163606 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4402fba0 con 0x7f3c5406ff20 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 5 monclient: authenticate success, global_id 4750 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 1 -- 10.20.193.6:0/2870163606 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44037440 con 0x7f3c5406ff20 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 10 monclient: get_monmap_and_config success 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 4 set_mon_vals no callback set 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c4b7fe640 10 monclient: dump: 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.528 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 10 monclient: shutdown 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2870163606 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5406ff20 msgr2=0x7f3c54070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2870163606 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5406ff20 0x7f3c54070300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c440098d0 tx=0x7f3c4402eed0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2870163606 shutdown_connections 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2870163606 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5406ff20 0x7f3c54070300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2870163606 >> 10.20.193.6:0/2870163606 conn(0x7f3c540cd780 msgr2=0x7f3c540cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2870163606 shutdown_connections 2026-02-22T23:24:58.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.526+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2870163606 wait complete. 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 10 monclient: build_initial_monmap 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 10 monclient: monmap: 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:58.530 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 1 librados: starting msgr at 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 1 librados: starting objecter 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 1 Processor -- start 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 1 -- start start 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 1 librados: setting wanted keys 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 1 librados: calling monclient init 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 10 monclient: init 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 10 monclient: _reopen_session rank -1 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.527+0000 7f3c5ae7c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5406ff20 0x7f3c540bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): picked mon.noname-a con 0x7f3c5406ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c540c99a0 0x7f3c5414ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): picked mon.noname-c con 0x7f3c540c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5415a5e0 0x7f3c541f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): picked mon.noname-b con 0x7f3c5415a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54150520 con 0x7f3c5406ff20 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c540c8a10 con 0x7f3c5415a5e0 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541506a0 con 0x7f3c540c99a0 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): _renew_subs 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c5ae7c640 10 monclient(hunting): authenticate will time out at 1173.255035s 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5406ff20 0x7f3c540bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5406ff20 0x7f3c540bf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:33262/0 (socket says 10.20.193.6:33262) 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 1 -- 10.20.193.6:0/2065658819 learned_addr learned my addr 10.20.193.6:0/2065658819 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:24:58.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 10 monclient(hunting): get_auth_request con 0x7f3c5406ff20 auth_method 0 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5415a5e0 0x7f3c541f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c4bfff640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c540c99a0 0x7f3c5414ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 10 monclient(hunting): get_auth_request con 0x7f3c5415a5e0 auth_method 0 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c4bfff640 10 monclient(hunting): get_auth_request con 0x7f3c540c99a0 auth_method 0 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c4bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c4bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c4bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.528+0000 7f3c593f2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c4bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c4bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:24:58.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c4bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 10 monclient(hunting): handle_auth_done global_id 4725 payload 824 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 1 -- 10.20.193.6:0/2065658819 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c540c99a0 msgr2=0x7f3c5414ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c540c99a0 0x7f3c5414ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 1 -- 10.20.193.6:0/2065658819 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5415a5e0 msgr2=0x7f3c541f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5415a5e0 0x7f3c541f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 10 monclient: _finish_hunting 0 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 1 monclient: found mon.noname-a 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.529+0000 7f3c58bf1640 1 -- 10.20.193.6:0/2065658819 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c540bde70 con 0x7f3c5406ff20 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.530+0000 7f3c58bf1640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5406ff20 0x7f3c540bf1f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3c4403a040 tx=0x7f3c440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.530+0000 7f3c497fa640 10 client.?.objecter ms_handle_connect 0x7f3c5406ff20 2026-02-22T23:24:58.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.530+0000 7f3c497fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 1 -- 10.20.193.6:0/2065658819 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44046020 con 0x7f3c5406ff20 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: dump: 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: _finish_auth 0 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: _check_auth_tickets 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 1 -- 10.20.193.6:0/2065658819 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c44004230 con 0x7f3c5406ff20 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 5 monclient: authenticate success, global_id 4725 2026-02-22T23:24:58.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 10 monclient: _renew_subs 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c540be150 con 0x7f3c5406ff20 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 1 -- 10.20.193.6:0/2065658819 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4403c040 con 0x7f3c5406ff20 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c497fa640 10 monclient: dump: 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c4affd640 4 set_mon_vals no callback set 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 10 client.4725.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 10 monclient: _renew_subs 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c540be710 con 0x7f3c5406ff20 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.532+0000 7f3c5ae7c640 1 librados: init done 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.533+0000 7f3c5ae7c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.533+0000 7f3c5ae7c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.533+0000 7f3c5ae7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:58.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.533+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c54070300 con 0x7f3c5406ff20 2026-02-22T23:24:58.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.544+0000 7f3c497fa640 1 -- 10.20.193.6:0/2065658819 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3c44053070 con 0x7f3c5406ff20 2026-02-22T23:24:58.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.544+0000 7f3c497fa640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c24078e50 0x7f3c2407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:24:58.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.544+0000 7f3c4bfff640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c24078e50 0x7f3c2407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:24:58.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.544+0000 7f3c4bfff640 10 monclient: get_auth_request con 0x7f3c24078e50 auth_method 0 2026-02-22T23:24:58.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.545+0000 7f3c4bfff640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c24078e50 0x7f3c2407b310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c540c1160 tx=0x7f3c3c007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:24:58.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.545+0000 7f3c497fa640 10 client.4725.objecter ms_handle_connect 0x7f3c24078e50 2026-02-22T23:24:58.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.545+0000 7f3c497fa640 1 -- 10.20.193.6:0/2065658819 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f3c4402fa60 con 0x7f3c5406ff20 2026-02-22T23:24:58.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.545+0000 7f3c497fa640 10 client.4725.objecter ms_dispatch2 0x7f3c54000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:24:58.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.545+0000 7f3c497fa640 3 client.4725.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:24:58.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.545+0000 7f3c497fa640 3 client.4725.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:24:58.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.545+0000 7f3c497fa640 20 client.4725.objecter dump_active .. 0 homeless 2026-02-22T23:24:58.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.640+0000 7f3c497fa640 1 -- 10.20.193.6:0/2065658819 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3c44050020 con 0x7f3c5406ff20 2026-02-22T23:24:58.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.640+0000 7f3c497fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:24:58.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.640+0000 7f3c497fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:24:58.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.741+0000 7f3c5ae7c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:24:58.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.741+0000 7f3c5ae7c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:58.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.741+0000 7f3c5ae7c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:24:58.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.741+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3c541573f0 con 0x7f3c5406ff20 2026-02-22T23:24:58.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.744+0000 7f3c497fa640 1 -- 10.20.193.6:0/2065658819 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3c44087090 con 0x7f3c5406ff20 2026-02-22T23:24:58.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.744+0000 7f3c497fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:24:58.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.744+0000 7f3c497fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:24:58.745 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:24:58.745 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 10 librados: watch_flush enter 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 10 librados: watch_flush exit 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 20 client.4725.objecter shutdown clearing up homeless session... 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 10 client.4725.objecter successfully canceled tick 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c24078e50 msgr2=0x7f3c2407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c24078e50 0x7f3c2407b310 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3c540c1160 tx=0x7f3c3c007ef0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 10 monclient: shutdown 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5406ff20 msgr2=0x7f3c540bf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5406ff20 0x7f3c540bf1f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3c4403a040 tx=0x7f3c440047c0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 shutdown_connections 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c24078e50 0x7f3c2407b310 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5415a5e0 0x7f3c541f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c540c99a0 0x7f3c5414ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 --2- 10.20.193.6:0/2065658819 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5406ff20 0x7f3c540bf1f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 >> 10.20.193.6:0/2065658819 conn(0x7f3c540cd780 msgr2=0x7f3c5414e620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:24:58.746 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 shutdown_connections 2026-02-22T23:24:58.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 -- 10.20.193.6:0/2065658819 wait complete. 2026-02-22T23:24:58.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:24:58.745+0000 7f3c5ae7c640 1 librados: shutdown 2026-02-22T23:25:01.756 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:01.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.822+0000 7fc46a5e4640 10 monclient: get_monmap_and_config 2026-02-22T23:25:01.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.822+0000 7fc46a5e4640 10 monclient: build_initial_monmap 2026-02-22T23:25:01.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.822+0000 7fc46a5e4640 10 monclient: monmap: 2026-02-22T23:25:01.823 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:01.823103+0000 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:01.823103+0000 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.822+0000 7fc46a5e4640 1 Processor -- start 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 1 -- start start 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient: init 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient: _add_conns ranks=[3,5,0,4,1,2] 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc46406ff20 0x7fc464070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient(hunting): picked mon.noname-a con 0x7fc46406ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:01.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc4640c99a0 0x7fc464159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient(hunting): picked mon.noname-c con 0x7fc4640c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc46415a410 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46415a860 con 0x7fc46415a410 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc464159a40 con 0x7fc46406ff20 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc4640c7e90 con 0x7fc4640c99a0 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc46a5e4640 10 monclient(hunting): authenticate will time out at 1176.550570s 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc4637fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc4640c99a0 0x7fc464159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.823+0000 7fc463fff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc46406ff20 0x7fc464070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc468b5a640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fc46415a410 0x7fc46414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:60536/0 (socket says 10.20.193.6:60536) 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc468b5a640 1 -- 10.20.193.6:0/791570127 learned_addr learned my addr 10.20.193.6:0/791570127 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc4637fe640 10 monclient(hunting): get_auth_request con 0x7fc4640c99a0 auth_method 0 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc4637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc4637fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc4637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc463fff640 10 monclient(hunting): get_auth_request con 0x7fc46406ff20 auth_method 0 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc463fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc463fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:01.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc463fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:01.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc4637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:01.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc4637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:01.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc4637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:01.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc463fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:01.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc463fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:01.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.824+0000 7fc463fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 10 monclient(hunting): handle_auth_done global_id 4835 payload 274 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 1 -- 10.20.193.6:0/791570127 >> v2:10.20.193.6:3300/0 conn(0x7fc46406ff20 msgr2=0x7fc464070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 1 --2- 10.20.193.6:0/791570127 >> v2:10.20.193.6:3300/0 conn(0x7fc46406ff20 0x7fc464070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 1 -- 10.20.193.6:0/791570127 >> v1:10.20.193.6:6789/0 conn(0x7fc46415a410 legacy=0x7fc46414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 10 monclient: _finish_hunting 0 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 1 monclient: found mon.noname-c 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:01.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.825+0000 7fc4637fe640 1 -- 10.20.193.6:0/791570127 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4640c8170 con 0x7fc4640c99a0 2026-02-22T23:25:01.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.826+0000 7fc4637fe640 1 --2- 10.20.193.6:0/791570127 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4640c99a0 0x7fc464159ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4540098a0 tx=0x7fc45402ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67e7840581a3a78 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 1 -- 10.20.193.6:0/791570127 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45403c070 con 0x7fc4640c99a0 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: dump: 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: _finish_auth 0 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 5 monclient: authenticate success, global_id 4835 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 1 -- 10.20.193.6:0/791570127 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45402fae0 con 0x7fc4640c99a0 2026-02-22T23:25:01.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 1 -- 10.20.193.6:0/791570127 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454037440 con 0x7fc4640c99a0 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 4 set_mon_vals no callback set 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc462ffd640 10 monclient: dump: 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 10 monclient: shutdown 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 -- 10.20.193.6:0/791570127 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4640c99a0 msgr2=0x7fc464159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/791570127 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4640c99a0 0x7fc464159ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc4540098a0 tx=0x7fc45402ee10 comp rx=0 tx=0).stop 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 -- 10.20.193.6:0/791570127 shutdown_connections 2026-02-22T23:25:01.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/791570127 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc4640c99a0 0x7fc464159ed0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:01.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/791570127 >> v2:10.20.193.6:3300/0 conn(0x7fc46406ff20 0x7fc464070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:01.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 -- 10.20.193.6:0/791570127 >> 10.20.193.6:0/791570127 conn(0x7fc4640cd780 msgr2=0x7fc4640cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:01.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 -- 10.20.193.6:0/791570127 shutdown_connections 2026-02-22T23:25:01.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.828+0000 7fc46a5e4640 1 -- 10.20.193.6:0/791570127 wait complete. 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 10 monclient: build_initial_monmap 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 10 monclient: monmap: 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 1 librados: starting msgr at 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 1 librados: starting objecter 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 1 Processor -- start 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 1 -- start start 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 1 librados: setting wanted keys 2026-02-22T23:25:01.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 1 librados: calling monclient init 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 10 monclient: init 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.829+0000 7fc46a5e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc46406ff20 0x7fc4641f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): picked mon.noname-b con 0x7fc46406ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4640c99a0 0x7fc46414ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): picked mon.noname-a con 0x7fc4640c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc46415a410 0x7fc464150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): picked mon.noname-c con 0x7fc46415a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc4640c8a10 con 0x7fc4640c99a0 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc4640c8730 con 0x7fc46406ff20 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc464150520 con 0x7fc46415a410 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc46a5e4640 10 monclient(hunting): authenticate will time out at 1176.557077s 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4640c99a0 0x7fc46414ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4640c99a0 0x7fc46414ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:33280/0 (socket says 10.20.193.6:33280) 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 1 -- 10.20.193.6:0/3355574449 learned_addr learned my addr 10.20.193.6:0/3355574449 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 10 monclient(hunting): get_auth_request con 0x7fc4640c99a0 auth_method 0 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:01.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc46415a410 0x7fc464150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc46406ff20 0x7fc4641f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 10 monclient(hunting): get_auth_request con 0x7fc46415a410 auth_method 0 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 10 monclient(hunting): get_auth_request con 0x7fc46406ff20 auth_method 0 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc4637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc468b5a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.830+0000 7fc463fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 10 monclient(hunting): handle_auth_done global_id 4838 payload 824 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 1 -- 10.20.193.6:0/3355574449 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc46406ff20 msgr2=0x7fc4641f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:01.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc46406ff20 0x7fc4641f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 1 -- 10.20.193.6:0/3355574449 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4640c99a0 msgr2=0x7fc46414ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4640c99a0 0x7fc46414ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 10 monclient: _finish_hunting 0 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 1 monclient: found mon.noname-c 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.832+0000 7fc468b5a640 1 -- 10.20.193.6:0/3355574449 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46414cf60 con 0x7fc46415a410 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.833+0000 7fc460ff9640 10 client.?.objecter ms_handle_connect 0x7fc46415a410 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.833+0000 7fc460ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:01.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.833+0000 7fc468b5a640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc46415a410 0x7fc464150900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc45800c9f0 tx=0x7fc45800cec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 1 -- 10.20.193.6:0/3355574449 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458017070 con 0x7fc46415a410 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: dump: 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:01.839 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: _finish_auth 0 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 5 monclient: authenticate success, global_id 4838 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 monclient: _renew_subs 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc464151480 con 0x7fc46415a410 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 client.4838.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 monclient: _renew_subs 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc464151a40 con 0x7fc46415a410 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 1 librados: init done 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 1 -- 10.20.193.6:0/3355574449 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc45800ab70 con 0x7fc46415a410 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 1 -- 10.20.193.6:0/3355574449 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc458021060 con 0x7fc46415a410 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc460ff9640 10 monclient: dump: 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:01.840 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc4627fc640 4 set_mon_vals no callback set 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:01.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.834+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc464070300 con 0x7fc46415a410 2026-02-22T23:25:01.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.945+0000 7fc460ff9640 1 -- 10.20.193.6:0/3355574449 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc458024080 con 0x7fc46415a410 2026-02-22T23:25:01.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.946+0000 7fc460ff9640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc434078e00 0x7fc43407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:01.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.946+0000 7fc463fff640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc434078e00 0x7fc43407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:01.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.946+0000 7fc463fff640 10 monclient: get_auth_request con 0x7fc434078e00 auth_method 0 2026-02-22T23:25:01.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.947+0000 7fc463fff640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc434078e00 0x7fc43407b2c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc450009c40 tx=0x7fc450009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:01.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.947+0000 7fc460ff9640 10 client.4838.objecter ms_handle_connect 0x7fc434078e00 2026-02-22T23:25:01.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.948+0000 7fc460ff9640 1 -- 10.20.193.6:0/3355574449 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc458097540 con 0x7fc46415a410 2026-02-22T23:25:01.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.948+0000 7fc460ff9640 10 client.4838.objecter ms_dispatch2 0x7fc464000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:01.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.948+0000 7fc460ff9640 3 client.4838.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:01.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.948+0000 7fc460ff9640 3 client.4838.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:01.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.948+0000 7fc460ff9640 20 client.4838.objecter dump_active .. 0 homeless 2026-02-22T23:25:01.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.955+0000 7fc460ff9640 1 -- 10.20.193.6:0/3355574449 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc458061c00 con 0x7fc46415a410 2026-02-22T23:25:01.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.955+0000 7fc460ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:01.956 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:01.955+0000 7fc460ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:02.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.054+0000 7fc46a5e4640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:02.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.054+0000 7fc46a5e4640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:02.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.054+0000 7fc46a5e4640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:02.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.054+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc4640c6270 con 0x7fc46415a410 2026-02-22T23:25:02.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.057+0000 7fc460ff9640 1 -- 10.20.193.6:0/3355574449 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc458065f00 con 0x7fc46415a410 2026-02-22T23:25:02.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.057+0000 7fc460ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:02.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.057+0000 7fc460ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:02.059 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:02.059 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:02.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 10 librados: watch_flush enter 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 10 librados: watch_flush exit 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 20 client.4838.objecter shutdown clearing up homeless session... 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 10 client.4838.objecter successfully canceled tick 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc434078e00 msgr2=0x7fc43407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc434078e00 0x7fc43407b2c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc450009c40 tx=0x7fc450009290 comp rx=0 tx=0).stop 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 10 monclient: shutdown 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc46415a410 msgr2=0x7fc464150900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc46415a410 0x7fc464150900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc45800c9f0 tx=0x7fc45800cec0 comp rx=0 tx=0).stop 2026-02-22T23:25:02.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 shutdown_connections 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc434078e00 0x7fc43407b2c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc46415a410 0x7fc464150900 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc4640c99a0 0x7fc46414ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 --2- 10.20.193.6:0/3355574449 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc46406ff20 0x7fc4641f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 >> 10.20.193.6:0/3355574449 conn(0x7fc4640cd780 msgr2=0x7fc4640c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 shutdown_connections 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 -- 10.20.193.6:0/3355574449 wait complete. 2026-02-22T23:25:02.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:02.058+0000 7fc46a5e4640 1 librados: shutdown 2026-02-22T23:25:05.069 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:05.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.137+0000 7f26a91e4640 10 monclient: get_monmap_and_config 2026-02-22T23:25:05.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.137+0000 7f26a91e4640 10 monclient: build_initial_monmap 2026-02-22T23:25:05.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.137+0000 7f26a91e4640 10 monclient: monmap: 2026-02-22T23:25:05.139 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:05.139 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:05.139 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:05.138385+0000 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:05.138385+0000 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:05.140 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:05.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 1 Processor -- start 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 1 -- start start 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient: init 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient: _add_conns ranks=[3,5,2,0,1,4] 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f26a41527d0 0x7f26a406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient(hunting): picked mon.noname-a con 0x7f26a41527d0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f26a4152bb0 0x7f26a40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient(hunting): picked mon.noname-c con 0x7f26a4152bb0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f26a40c5fc0 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a406e300 con 0x7f26a40c5fc0 2026-02-22T23:25:05.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f26a406f1f0 con 0x7f26a41527d0 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26a40c6470 con 0x7f26a4152bb0 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.138+0000 7f26a91e4640 10 monclient(hunting): authenticate will time out at 1179.865879s 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f26a41527d0 0x7f26a406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2575640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f26a4152bb0 0x7f26a40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f26a41527d0 0x7f26a406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:33282/0 (socket says 10.20.193.6:33282) 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 1 -- 10.20.193.6:0/1106716184 learned_addr learned my addr 10.20.193.6:0/1106716184 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 10 monclient(hunting): get_auth_request con 0x7f26a41527d0 auth_method 0 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:05.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2575640 10 monclient(hunting): get_auth_request con 0x7f26a4152bb0 auth_method 0 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.139+0000 7f26a2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 10 monclient(hunting): handle_auth_done global_id 4737 payload 274 2026-02-22T23:25:05.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 1 -- 10.20.193.6:0/1106716184 >> v2:10.20.193.80:3300/0 conn(0x7f26a4152bb0 msgr2=0x7f26a40c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 1 --2- 10.20.193.6:0/1106716184 >> v2:10.20.193.80:3300/0 conn(0x7f26a4152bb0 0x7f26a40c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 1 -- 10.20.193.6:0/1106716184 >> v1:10.20.193.80:6789/0 conn(0x7f26a40c5fc0 legacy=0x7f26a40c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 10 monclient: _finish_hunting 0 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 1 monclient: found mon.noname-a 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.140+0000 7f26a2d76640 1 -- 10.20.193.6:0/1106716184 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a406e5e0 con 0x7f26a41527d0 2026-02-22T23:25:05.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.141+0000 7f26a2d76640 1 --2- 10.20.193.6:0/1106716184 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a406da60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26a40750e0 tx=0x7f269802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1cf1536ddd36db server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:05.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 1 -- 10.20.193.6:0/1106716184 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269803c070 con 0x7f26a41527d0 2026-02-22T23:25:05.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: dump: 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:05.154 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: _finish_auth 0 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: _check_auth_tickets 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 1 -- 10.20.193.6:0/1106716184 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269802fbe0 con 0x7f26a41527d0 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 5 monclient: authenticate success, global_id 4737 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 1 -- 10.20.193.6:0/1106716184 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698037440 con 0x7f26a41527d0 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:05.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a1d74640 10 monclient: dump: 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:05.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 4 set_mon_vals no callback set 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 10 monclient: shutdown 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 -- 10.20.193.6:0/1106716184 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 msgr2=0x7f26a406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 --2- 10.20.193.6:0/1106716184 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a406da60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26a40750e0 tx=0x7f269802eed0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 -- 10.20.193.6:0/1106716184 shutdown_connections 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 --2- 10.20.193.6:0/1106716184 >> v2:10.20.193.80:3300/0 conn(0x7f26a4152bb0 0x7f26a40c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 --2- 10.20.193.6:0/1106716184 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a406da60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 -- 10.20.193.6:0/1106716184 >> 10.20.193.6:0/1106716184 conn(0x7f26a40cd780 msgr2=0x7f26a40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 -- 10.20.193.6:0/1106716184 shutdown_connections 2026-02-22T23:25:05.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.143+0000 7f26a91e4640 1 -- 10.20.193.6:0/1106716184 wait complete. 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 10 monclient: build_initial_monmap 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 10 monclient: monmap: 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:05.158 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 1 librados: starting msgr at 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 1 librados: starting objecter 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 1 Processor -- start 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 1 -- start start 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 1 librados: setting wanted keys 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 1 librados: calling monclient init 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.144+0000 7f26a91e4640 10 monclient: init 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:05.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f26a40c5fc0 0x7f26a41f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): picked mon.noname-c con 0x7f26a40c5fc0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): picked mon.noname-a con 0x7f26a41527d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f26a4152bb0 0x7f26a41ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): picked mon.noname-b con 0x7f26a4152bb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f26a406ee80 con 0x7f26a41527d0 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f26a406eba0 con 0x7f26a4152bb0 2026-02-22T23:25:05.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26a406dfa0 con 0x7f26a40c5fc0 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a91e4640 10 monclient(hunting): authenticate will time out at 1179.872217s 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2d76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f26a40c5fc0 0x7f26a41f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:33290/0 (socket says 10.20.193.6:33290) 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 1 -- 10.20.193.6:0/2478438407 learned_addr learned my addr 10.20.193.6:0/2478438407 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 10 monclient(hunting): get_auth_request con 0x7f26a41527d0 auth_method 0 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:05.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a3577640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f26a4152bb0 0x7f26a41ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2d76640 10 monclient(hunting): get_auth_request con 0x7f26a40c5fc0 auth_method 0 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a3577640 10 monclient(hunting): get_auth_request con 0x7f26a4152bb0 auth_method 0 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a3577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a3577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a3577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.145+0000 7f26a2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.146+0000 7f26a2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:05.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.146+0000 7f26a2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.146+0000 7f26a2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.146+0000 7f26a3577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.146+0000 7f26a3577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.146+0000 7f26a3577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 10 monclient(hunting): handle_auth_done global_id 4740 payload 824 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 1 -- 10.20.193.6:0/2478438407 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f26a40c5fc0 msgr2=0x7f26a41f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f26a40c5fc0 0x7f26a41f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 1 -- 10.20.193.6:0/2478438407 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f26a4152bb0 msgr2=0x7f26a41ef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f26a4152bb0 0x7f26a41ef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 10 monclient: _finish_hunting 0 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 1 monclient: found mon.noname-a 2026-02-22T23:25:05.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.147+0000 7f26a2575640 1 -- 10.20.193.6:0/2478438407 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a41f3800 con 0x7f26a41527d0 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.148+0000 7f26a2575640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a4157a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e890 tx=0x7f268c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.148+0000 7f26837fe640 10 client.?.objecter ms_handle_connect 0x7f26a41527d0 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.148+0000 7f26837fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 1 -- 10.20.193.6:0/2478438407 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c017070 con 0x7f26a41527d0 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: dump: 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.164 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: _finish_auth 0 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: _check_auth_tickets 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 5 monclient: authenticate success, global_id 4740 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 1 -- 10.20.193.6:0/2478438407 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c00c940 con 0x7f26a41527d0 2026-02-22T23:25:05.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 1 -- 10.20.193.6:0/2478438407 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c00cd30 con 0x7f26a41527d0 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 10 monclient: _renew_subs 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a1573640 4 set_mon_vals no callback set 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a41f3ae0 con 0x7f26a41527d0 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26837fe640 10 monclient: dump: 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:05.166 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 10 client.4740.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 10 monclient: _renew_subs 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a41f40a0 con 0x7f26a41527d0 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.149+0000 7f26a91e4640 1 librados: init done 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.150+0000 7f26a91e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.150+0000 7f26a91e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:05.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.150+0000 7f26a91e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:05.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.150+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a406da60 con 0x7f26a41527d0 2026-02-22T23:25:05.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.251+0000 7f26837fe640 1 -- 10.20.193.6:0/2478438407 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f268c013470 con 0x7f26a41527d0 2026-02-22T23:25:05.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.251+0000 7f26837fe640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2670078e00 0x7f267007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:05.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.252+0000 7f26a2d76640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2670078e00 0x7f267007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:05.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.252+0000 7f26a2d76640 10 monclient: get_auth_request con 0x7f2670078e00 auth_method 0 2026-02-22T23:25:05.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.252+0000 7f26837fe640 10 client.4740.objecter ms_handle_connect 0x7f2670078e00 2026-02-22T23:25:05.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.252+0000 7f26a2d76640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2670078e00 0x7f267007b2c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26a4158940 tx=0x7f2698039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:05.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 1 -- 10.20.193.6:0/2478438407 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f268c05e3e0 con 0x7f26a41527d0 2026-02-22T23:25:05.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 10 client.4740.objecter ms_dispatch2 0x7f26a4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:05.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 3 client.4740.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:05.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 3 client.4740.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:05.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 20 client.4740.objecter dump_active .. 0 homeless 2026-02-22T23:25:05.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 1 -- 10.20.193.6:0/2478438407 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f268c0623a0 con 0x7f26a41527d0 2026-02-22T23:25:05.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:05.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.259+0000 7f26837fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:05.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.361+0000 7f26a91e4640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:05.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.361+0000 7f26a91e4640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:05.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.361+0000 7f26a91e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:05.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.361+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f26a41518d0 con 0x7f26a41527d0 2026-02-22T23:25:05.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.364+0000 7f26837fe640 1 -- 10.20.193.6:0/2478438407 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f268c0666a0 con 0x7f26a41527d0 2026-02-22T23:25:05.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.364+0000 7f26837fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:05.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.364+0000 7f26837fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:05.366 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:05.366 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 10 librados: watch_flush enter 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 10 librados: watch_flush exit 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 20 client.4740.objecter shutdown clearing up homeless session... 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 10 client.4740.objecter successfully canceled tick 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2670078e00 msgr2=0x7f267007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2670078e00 0x7f267007b2c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26a4158940 tx=0x7f2698039040 comp rx=0 tx=0).stop 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 10 monclient: shutdown 2026-02-22T23:25:05.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 msgr2=0x7f26a4157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a4157a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f268c00e890 tx=0x7f268c00ed60 comp rx=0 tx=0).stop 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 shutdown_connections 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2670078e00 0x7f267007b2c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f26a4152bb0 0x7f26a41ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f26a41527d0 0x7f26a4157a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 --2- 10.20.193.6:0/2478438407 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f26a40c5fc0 0x7f26a41f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 >> 10.20.193.6:0/2478438407 conn(0x7f26a40cd780 msgr2=0x7f26a406fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 shutdown_connections 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 -- 10.20.193.6:0/2478438407 wait complete. 2026-02-22T23:25:05.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:05.365+0000 7f26a91e4640 1 librados: shutdown 2026-02-22T23:25:08.376 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.446+0000 7f7523b96640 10 monclient: get_monmap_and_config 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.446+0000 7f7523b96640 10 monclient: build_initial_monmap 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.446+0000 7f7523b96640 10 monclient: monmap: 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:08.446739+0000 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:08.446739+0000 2026-02-22T23:25:08.448 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.446+0000 7f7523b96640 1 Processor -- start 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.446+0000 7f7523b96640 1 -- start start 2026-02-22T23:25:08.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.446+0000 7f7523b96640 10 monclient: init 2026-02-22T23:25:08.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:08.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient: _add_conns ranks=[4,3,0,2,1,5] 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f751c06ff20 0x7f751c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient(hunting): picked mon.noname-b con 0x7f751c06ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f751c0c99a0 0x7f751c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient(hunting): picked mon.noname-a con 0x7f751c0c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f751c15a410 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f751c15a860 con 0x7f751c15a410 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f751c159a40 con 0x7f751c0c99a0 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f751c0c7e90 con 0x7f751c06ff20 2026-02-22T23:25:08.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f7523b96640 10 monclient(hunting): authenticate will time out at 1183.174114s 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752190b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f751c06ff20 0x7f751c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752110a640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f751c0c99a0 0x7f751c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752210c640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f751c15a410 0x7f751c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:42948/0 (socket says 10.20.193.6:42948) 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752210c640 1 -- 10.20.193.6:0/1661276862 learned_addr learned my addr 10.20.193.6:0/1661276862 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752190b640 10 monclient(hunting): get_auth_request con 0x7f751c06ff20 auth_method 0 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752190b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752190b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.447+0000 7f752190b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752110a640 10 monclient(hunting): get_auth_request con 0x7f751c0c99a0 auth_method 0 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752110a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:08.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752110a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752110a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752190b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752190b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752190b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752110a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752110a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.448+0000 7f752110a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 10 monclient(hunting): handle_auth_done global_id 4765 payload 274 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 1 -- 10.20.193.6:0/1661276862 >> v2:10.20.193.6:3300/0 conn(0x7f751c0c99a0 msgr2=0x7f751c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 1 --2- 10.20.193.6:0/1661276862 >> v2:10.20.193.6:3300/0 conn(0x7f751c0c99a0 0x7f751c159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 1 -- 10.20.193.6:0/1661276862 >> v1:10.20.193.6:6789/0 conn(0x7f751c15a410 legacy=0x7f751c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 10 monclient: _finish_hunting 0 2026-02-22T23:25:08.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 1 monclient: found mon.noname-b 2026-02-22T23:25:08.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:08.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:25:08.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.449+0000 7f752190b640 1 -- 10.20.193.6:0/1661276862 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c0c8170 con 0x7f751c06ff20 2026-02-22T23:25:08.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.450+0000 7f752190b640 1 --2- 10.20.193.6:0/1661276862 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c070300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f750c0098a0 tx=0x7f750c02ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9fe5cc98d2c076 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:08.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.451+0000 7f7520909640 1 -- 10.20.193.6:0/1661276862 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c03c070 con 0x7f751c06ff20 2026-02-22T23:25:08.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:08.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:25:08.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: dump: 2026-02-22T23:25:08.462 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:08.462 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: _finish_auth 0 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: _check_auth_tickets 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:08.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 5 monclient: authenticate success, global_id 4765 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 1 -- 10.20.193.6:0/1661276862 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c02fae0 con 0x7f751c06ff20 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 1 -- 10.20.193.6:0/1661276862 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c037440 con 0x7f751c06ff20 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7520909640 10 monclient: dump: 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:08.464 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 4 set_mon_vals no callback set 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 10 monclient: shutdown 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 -- 10.20.193.6:0/1661276862 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 msgr2=0x7f751c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 --2- 10.20.193.6:0/1661276862 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c070300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f750c0098a0 tx=0x7f750c02ee10 comp rx=0 tx=0).stop 2026-02-22T23:25:08.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 -- 10.20.193.6:0/1661276862 shutdown_connections 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 --2- 10.20.193.6:0/1661276862 >> v2:10.20.193.6:3300/0 conn(0x7f751c0c99a0 0x7f751c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 --2- 10.20.193.6:0/1661276862 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c070300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 -- 10.20.193.6:0/1661276862 >> 10.20.193.6:0/1661276862 conn(0x7f751c0cd780 msgr2=0x7f751c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 -- 10.20.193.6:0/1661276862 shutdown_connections 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 1 -- 10.20.193.6:0/1661276862 wait complete. 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 10 monclient: build_initial_monmap 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.452+0000 7f7523b96640 10 monclient: monmap: 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:08.466 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 librados: starting msgr at 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 librados: starting objecter 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 Processor -- start 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 -- start start 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 librados: setting wanted keys 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 librados: calling monclient init 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient: init 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:08.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): picked mon.noname-b con 0x7f751c06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f751c0c99a0 0x7f751c14cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): picked mon.noname-a con 0x7f751c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f751c15a410 0x7f751c1f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): picked mon.noname-c con 0x7f751c15a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f751c0c8a10 con 0x7f751c0c99a0 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f751c0c8730 con 0x7f751c06ff20 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:08.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f751c150520 con 0x7f751c15a410 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f7523b96640 10 monclient(hunting): authenticate will time out at 1183.180644s 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752110a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f751c0c99a0 0x7f751c14cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752190b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752110a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f751c0c99a0 0x7f751c14cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43678/0 (socket says 10.20.193.6:43678) 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752110a640 1 -- 10.20.193.6:0/2218418948 learned_addr learned my addr 10.20.193.6:0/2218418948 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752110a640 10 monclient(hunting): get_auth_request con 0x7f751c0c99a0 auth_method 0 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752110a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752110a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.453+0000 7f752110a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752190b640 10 monclient(hunting): get_auth_request con 0x7f751c06ff20 auth_method 0 2026-02-22T23:25:08.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752190b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752190b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752190b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f751c15a410 0x7f751c1f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752110a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752110a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752110a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752190b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752190b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752190b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 10 monclient(hunting): get_auth_request con 0x7f751c15a410 auth_method 0 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:08.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.454+0000 7f752210c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 10 monclient(hunting): handle_auth_done global_id 4768 payload 824 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 1 -- 10.20.193.6:0/2218418948 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f751c15a410 msgr2=0x7f751c1f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f751c15a410 0x7f751c1f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 1 -- 10.20.193.6:0/2218418948 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f751c0c99a0 msgr2=0x7f751c14cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f751c0c99a0 0x7f751c14cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 10 monclient: _finish_hunting 0 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 1 monclient: found mon.noname-b 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:25:08.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.455+0000 7f752190b640 1 -- 10.20.193.6:0/2218418948 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c154440 con 0x7f751c06ff20 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.456+0000 7f752190b640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c156da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f751c159bc0 tx=0x7f750c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.456+0000 7f750a7fc640 10 client.?.objecter ms_handle_connect 0x7f751c06ff20 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.456+0000 7f750a7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 1 -- 10.20.193.6:0/2218418948 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c046020 con 0x7f751c06ff20 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: dump: 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.472 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 1 -- 10.20.193.6:0/2218418948 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c042670 con 0x7f751c06ff20 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 1 -- 10.20.193.6:0/2218418948 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c03c040 con 0x7f751c06ff20 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 5 monclient: authenticate success, global_id 4768 2026-02-22T23:25:08.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750bfff640 4 set_mon_vals no callback set 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 monclient: _renew_subs 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c154720 con 0x7f751c06ff20 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f750a7fc640 10 monclient: dump: 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:08.474 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 client.4768.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 monclient: _renew_subs 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c154ce0 con 0x7f751c06ff20 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 1 librados: init done 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:08.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.458+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751c070300 con 0x7f751c06ff20 2026-02-22T23:25:08.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.563+0000 7f750a7fc640 1 -- 10.20.193.6:0/2218418948 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f750c04b440 con 0x7f751c06ff20 2026-02-22T23:25:08.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.564+0000 7f750a7fc640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74ec078e50 0x7f74ec07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:08.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f752110a640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74ec078e50 0x7f74ec07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:08.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f752110a640 10 monclient: get_auth_request con 0x7f74ec078e50 auth_method 0 2026-02-22T23:25:08.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f750a7fc640 1 -- 10.20.193.6:0/2218418948 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f750c02fa20 con 0x7f751c06ff20 2026-02-22T23:25:08.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f750a7fc640 10 client.4768.objecter ms_dispatch2 0x7f751c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:08.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f750a7fc640 3 client.4768.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:08.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f750a7fc640 3 client.4768.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:08.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f752110a640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74ec078e50 0x7f74ec07b310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f751c0c43e0 tx=0x7f7510009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:08.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f750a7fc640 20 client.4768.objecter dump_active .. 0 homeless 2026-02-22T23:25:08.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.565+0000 7f750a7fc640 10 client.4768.objecter ms_handle_connect 0x7f74ec078e50 2026-02-22T23:25:08.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.572+0000 7f750a7fc640 1 -- 10.20.193.6:0/2218418948 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f750c050080 con 0x7f751c06ff20 2026-02-22T23:25:08.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.572+0000 7f750a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:08.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.572+0000 7f750a7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:08.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.673+0000 7f7523b96640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:08.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.673+0000 7f7523b96640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:08.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.673+0000 7f7523b96640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:08.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.673+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f751c1587f0 con 0x7f751c06ff20 2026-02-22T23:25:08.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.676+0000 7f750a7fc640 1 -- 10.20.193.6:0/2218418948 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f750c0867e0 con 0x7f751c06ff20 2026-02-22T23:25:08.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.676+0000 7f750a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:08.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.676+0000 7f750a7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:08.677 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:08.678 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:08.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 10 librados: watch_flush enter 2026-02-22T23:25:08.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 10 librados: watch_flush exit 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 20 client.4768.objecter shutdown clearing up homeless session... 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 10 client.4768.objecter successfully canceled tick 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74ec078e50 msgr2=0x7f74ec07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74ec078e50 0x7f74ec07b310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f751c0c43e0 tx=0x7f7510009290 comp rx=0 tx=0).stop 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 10 monclient: shutdown 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 msgr2=0x7f751c156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c156da0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f751c159bc0 tx=0x7f750c0047c0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 shutdown_connections 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74ec078e50 0x7f74ec07b310 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f751c15a410 0x7f751c1f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f751c0c99a0 0x7f751c14cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 --2- 10.20.193.6:0/2218418948 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f751c06ff20 0x7f751c156da0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:08.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 >> 10.20.193.6:0/2218418948 conn(0x7f751c0cd780 msgr2=0x7f751c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:08.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 shutdown_connections 2026-02-22T23:25:08.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 -- 10.20.193.6:0/2218418948 wait complete. 2026-02-22T23:25:08.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:08.677+0000 7f7523b96640 1 librados: shutdown 2026-02-22T23:25:11.688 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.756+0000 7f94d7340640 10 monclient: get_monmap_and_config 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.756+0000 7f94d7340640 10 monclient: build_initial_monmap 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.756+0000 7f94d7340640 10 monclient: monmap: 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:11.756785+0000 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:11.756785+0000 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:11.757 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.756+0000 7f94d7340640 1 Processor -- start 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.756+0000 7f94d7340640 1 -- start start 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.756+0000 7f94d7340640 10 monclient: init 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 10 monclient: _add_conns ranks=[1,0,2,3,4,5] 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f94d01527d0 addr v1:10.20.193.40:6789/0 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f94d0152bb0 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f94d00c6190 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d006e300 con 0x7f94d0152bb0 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d006e5e0 con 0x7f94d01527d0 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94d006e8c0 con 0x7f94d00c6190 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d7340640 10 monclient(hunting): authenticate will time out at 1186.484276s 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d48b4640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f94d0152bb0 0x7f94d00c5a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:42956/0 (socket says 10.20.193.6:42956) 2026-02-22T23:25:11.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.757+0000 7f94d48b4640 1 -- 10.20.193.6:0/3529320797 learned_addr learned my addr 10.20.193.6:0/3529320797 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:11.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.758+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829517003 0 0) 0x7f94d006e300 con 0x7f94d0152bb0 2026-02-22T23:25:11.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.758+0000 7f94bffff640 10 monclient(hunting): _negotiate 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.758+0000 7f94bffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.758+0000 7f94bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.758+0000 7f94bffff640 10 monclient(hunting): my global_id is 4752 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.758+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94b8003620 con 0x7f94d0152bb0 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982608419 0 0) 0x7f94d006e5e0 con 0x7f94d01527d0 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): _negotiate 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): my global_id is 4771 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94d006e300 con 0x7f94d01527d0 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465693531 0 0) 0x7f94d006e8c0 con 0x7f94d00c6190 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): _negotiate 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 10 monclient(hunting): my global_id is 4853 2026-02-22T23:25:11.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.759+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94d006e5e0 con 0x7f94d00c6190 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1109012126 0 0) 0x7f94b8003620 con 0x7f94d0152bb0 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 >> v1:10.20.193.80:6789/0 conn(0x7f94d00c6190 legacy=0x7f94d00c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 >> v1:10.20.193.40:6789/0 conn(0x7f94d01527d0 legacy=0x7f94d006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d006eba0 con 0x7f94d0152bb0 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: _finish_auth 0 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: _check_auth_tickets 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f94c4002f00 con 0x7f94d0152bb0 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94d7340640 5 monclient: authenticate success, global_id 4752 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94d7340640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: dump: 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 >> v1:10.20.193.6:6789/0 conn(0x7f94d0152bb0 legacy=0x7f94d00c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8003fe0 0x7f94b8004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): picked mon.b con 0x7f94b8003fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94b8004e40 0x7f94b80052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): picked mon.a con 0x7f94b8004e40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94b8009ab0 0x7f94b800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:11.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): picked mon.c con 0x7f94b8009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94b800c570 con 0x7f94b8004e40 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94b800e980 con 0x7f94b8003fe0 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f94b800eb80 con 0x7f94b8009ab0 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94bffff640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.760+0000 7f94d7340640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.761+0000 7f94d58b6640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94b8009ab0 0x7f94b800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.761+0000 7f94d58b6640 10 monclient(hunting): get_auth_request con 0x7f94b8009ab0 auth_method 0 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.761+0000 7f94d58b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.761+0000 7f94d58b6640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.761+0000 7f94d58b6640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.761+0000 7f94d50b5640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8003fe0 0x7f94b8004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d50b5640 10 monclient(hunting): get_auth_request con 0x7f94b8003fe0 auth_method 0 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d50b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d50b5640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d50b5640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d58b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d58b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d58b6640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d50b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:11.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d50b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d50b5640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94b8004e40 0x7f94b80052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 10 monclient(hunting): get_auth_request con 0x7f94b8004e40 auth_method 0 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.762+0000 7f94d48b4640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 10 monclient(hunting): handle_auth_done global_id 4752 payload 293 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 1 -- 10.20.193.6:0/3529320797 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8003fe0 msgr2=0x7f94b8004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8003fe0 0x7f94b8004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 1 -- 10.20.193.6:0/3529320797 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94b8004e40 msgr2=0x7f94b80052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94b8004e40 0x7f94b80052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 10 monclient: _finish_hunting 0 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 1 monclient: found mon.c 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 1 -- 10.20.193.6:0/3529320797 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b8003620 con 0x7f94b8009ab0 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 10 monclient: _finish_auth 0 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 10 monclient: _check_auth_tickets 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:11.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.763+0000 7f94d58b6640 1 -- 10.20.193.6:0/3529320797 reap_dead start 2026-02-22T23:25:11.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.764+0000 7f94d58b6640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94b8009ab0 0x7f94b800bea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c320 tx=0x7f94cc02f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2a84dfdc4bd5ed server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003860 con 0x7f94b8009ab0 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 10 monclient: dump: 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc003c70 con 0x7f94b8009ab0 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 1 -- 10.20.193.6:0/3529320797 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc005310 con 0x7f94b8009ab0 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94bffff640 10 monclient: dump: 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:11.767 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 4 set_mon_vals no callback set 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 10 monclient: shutdown 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 1 -- 10.20.193.6:0/3529320797 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94b8009ab0 msgr2=0x7f94b800bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94b8009ab0 0x7f94b800bea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c320 tx=0x7f94cc02f610 comp rx=0 tx=0).stop 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 1 -- 10.20.193.6:0/3529320797 shutdown_connections 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 1 --2- 10.20.193.6:0/3529320797 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94b8009ab0 0x7f94b800bea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 1 -- 10.20.193.6:0/3529320797 >> 10.20.193.6:0/3529320797 conn(0x7f94d00cd780 msgr2=0x7f94d00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 1 -- 10.20.193.6:0/3529320797 shutdown_connections 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.766+0000 7f94d7340640 1 -- 10.20.193.6:0/3529320797 wait complete. 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient: build_initial_monmap 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient: monmap: 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:11.768 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 librados: starting msgr at 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 librados: starting objecter 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 Processor -- start 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 -- start start 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 librados: setting wanted keys 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 librados: calling monclient init 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient: init 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8009ab0 0x7f94d0074080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): picked mon.noname-b con 0x7f94b8009ab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94d01efe00 0x7f94d0144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): picked mon.noname-a con 0x7f94d01efe00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94d00cc560 0x7f94d0146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): picked mon.noname-c con 0x7f94d00cc560 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f94d00c5a80 con 0x7f94d01efe00 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f94d006f1f0 con 0x7f94b8009ab0 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f94d00c5c00 con 0x7f94d00cc560 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.767+0000 7f94d7340640 10 monclient(hunting): authenticate will time out at 1186.494993s 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94d01efe00 0x7f94d0144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:11.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94d00cc560 0x7f94d0146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94d01efe00 0x7f94d0144960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43698/0 (socket says 10.20.193.6:43698) 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 1 -- 10.20.193.6:0/1936974170 learned_addr learned my addr 10.20.193.6:0/1936974170 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 10 monclient(hunting): get_auth_request con 0x7f94d01efe00 auth_method 0 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 10 monclient(hunting): get_auth_request con 0x7f94d00cc560 auth_method 0 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:11.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d50b5640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8009ab0 0x7f94d0074080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d58b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d50b5640 10 monclient(hunting): get_auth_request con 0x7f94b8009ab0 auth_method 0 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d50b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d50b5640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d50b5640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.768+0000 7f94d48b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d50b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:11.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d50b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:11.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d50b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 10 monclient(hunting): handle_auth_done global_id 4856 payload 824 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 1 -- 10.20.193.6:0/1936974170 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8009ab0 msgr2=0x7f94d0074080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8009ab0 0x7f94d0074080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 1 -- 10.20.193.6:0/1936974170 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94d01efe00 msgr2=0x7f94d0144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94d01efe00 0x7f94d0144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 10 monclient: _finish_hunting 0 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 1 monclient: found mon.noname-c 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.769+0000 7f94d58b6640 1 -- 10.20.193.6:0/1936974170 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d00725b0 con 0x7f94d00cc560 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.771+0000 7f94d58b6640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94d00cc560 0x7f94d0146600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94d00c55f0 tx=0x7f94cc002e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.771+0000 7f94bdffb640 10 client.?.objecter ms_handle_connect 0x7f94d00cc560 2026-02-22T23:25:11.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.771+0000 7f94bdffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 1 -- 10.20.193.6:0/1936974170 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc003860 con 0x7f94d00cc560 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: dump: 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:11.775 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: _finish_auth 0 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: _check_auth_tickets 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 1 -- 10.20.193.6:0/1936974170 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94cc003e20 con 0x7f94d00cc560 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 1 -- 10.20.193.6:0/1936974170 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94cc036850 con 0x7f94d00cc560 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bf7fe640 4 set_mon_vals no callback set 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94bdffb640 10 monclient: dump: 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:11.776 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 5 monclient: authenticate success, global_id 4856 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 10 monclient: _renew_subs 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d0072890 con 0x7f94d00cc560 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 10 client.4856.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 10 monclient: _renew_subs 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d0072e50 con 0x7f94d00cc560 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.772+0000 7f94d7340640 1 librados: init done 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.773+0000 7f94d7340640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.773+0000 7f94d7340640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.773+0000 7f94d7340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:11.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.773+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d006da60 con 0x7f94d00cc560 2026-02-22T23:25:11.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.784+0000 7f94bdffb640 1 -- 10.20.193.6:0/1936974170 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f94cc0369f0 con 0x7f94d00cc560 2026-02-22T23:25:11.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.784+0000 7f94bdffb640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f94a0078e80 0x7f94a007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:11.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.784+0000 7f94d50b5640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f94a0078e80 0x7f94a007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:11.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.784+0000 7f94d50b5640 10 monclient: get_auth_request con 0x7f94a0078e80 auth_method 0 2026-02-22T23:25:11.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.785+0000 7f94bdffb640 10 client.4856.objecter ms_handle_connect 0x7f94a0078e80 2026-02-22T23:25:11.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.785+0000 7f94d50b5640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f94a0078e80 0x7f94a007b340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94c0002910 tx=0x7f94c0007da0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:11.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.785+0000 7f94bdffb640 1 -- 10.20.193.6:0/1936974170 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f94cc036d10 con 0x7f94d00cc560 2026-02-22T23:25:11.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.785+0000 7f94bdffb640 10 client.4856.objecter ms_dispatch2 0x7f94d0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:11.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.785+0000 7f94bdffb640 3 client.4856.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:11.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.785+0000 7f94bdffb640 3 client.4856.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:11.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.785+0000 7f94bdffb640 20 client.4856.objecter dump_active .. 0 homeless 2026-02-22T23:25:11.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.885+0000 7f94bdffb640 1 -- 10.20.193.6:0/1936974170 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f94cc049060 con 0x7f94d00cc560 2026-02-22T23:25:11.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.885+0000 7f94bdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:11.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.885+0000 7f94bdffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:11.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.987+0000 7f94d7340640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:11.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.987+0000 7f94d7340640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:11.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.987+0000 7f94d7340640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:11.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.987+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f94d00c7f90 con 0x7f94d00cc560 2026-02-22T23:25:11.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.989+0000 7f94bdffb640 1 -- 10.20.193.6:0/1936974170 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f94cc086ce0 con 0x7f94d00cc560 2026-02-22T23:25:11.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.989+0000 7f94bdffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:11.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.989+0000 7f94bdffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:11.991 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:11.991 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:11.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.990+0000 7f94d7340640 10 librados: watch_flush enter 2026-02-22T23:25:11.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 10 librados: watch_flush exit 2026-02-22T23:25:11.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 20 client.4856.objecter shutdown clearing up homeless session... 2026-02-22T23:25:11.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 10 client.4856.objecter successfully canceled tick 2026-02-22T23:25:11.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f94a0078e80 msgr2=0x7f94a007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f94a0078e80 0x7f94a007b340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94c0002910 tx=0x7f94c0007da0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 10 monclient: shutdown 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94d00cc560 msgr2=0x7f94d0146600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94d00cc560 0x7f94d0146600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f94d00c55f0 tx=0x7f94cc002e40 comp rx=0 tx=0).stop 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 shutdown_connections 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f94a0078e80 0x7f94a007b340 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f94d00cc560 0x7f94d0146600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f94d01efe00 0x7f94d0144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 --2- 10.20.193.6:0/1936974170 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f94b8009ab0 0x7f94d0074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:11.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 >> 10.20.193.6:0/1936974170 conn(0x7f94d00cd780 msgr2=0x7f94d00c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:11.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 shutdown_connections 2026-02-22T23:25:11.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 -- 10.20.193.6:0/1936974170 wait complete. 2026-02-22T23:25:11.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:11.991+0000 7f94d7340640 1 librados: shutdown 2026-02-22T23:25:13.034 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:57.283 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.035 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:57.285 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.044 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:03:57.294 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.061 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:57.308 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.062 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:57.273 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.062 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:57.308 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.063 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:57.274 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.072 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:03:57.318 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:13.072 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:03:57.283 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:15.001 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:15.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.068+0000 7f2ce4508640 10 monclient: get_monmap_and_config 2026-02-22T23:25:15.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.068+0000 7f2ce4508640 10 monclient: build_initial_monmap 2026-02-22T23:25:15.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.068+0000 7f2ce4508640 10 monclient: monmap: 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:15.070147+0000 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:15.070147+0000 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:15.071 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:15.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 1 Processor -- start 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 1 -- start start 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient: init 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient: _add_conns ranks=[3,5,2,0,1,4] 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f2cdc06ff20 0x7f2cdc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient(hunting): picked mon.noname-a con 0x7f2cdc06ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f2cdc0c99a0 0x7f2cdc159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient(hunting): picked mon.noname-c con 0x7f2cdc0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f2cdc15a410 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cdc15a860 con 0x7f2cdc15a410 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f2cdc159a40 con 0x7f2cdc06ff20 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2cdc0c7e90 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce4508640 10 monclient(hunting): authenticate will time out at 1189.797610s 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.069+0000 7f2ce227d640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f2cdc06ff20 0x7f2cdc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f2cdc06ff20 0x7f2cdc070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:43704/0 (socket says 10.20.193.6:43704) 2026-02-22T23:25:15.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 1 -- 10.20.193.6:0/3509690853 learned_addr learned my addr 10.20.193.6:0/3509690853 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 10 monclient(hunting): get_auth_request con 0x7f2cdc06ff20 auth_method 0 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 1 --2- 10.20.193.6:0/3509690853 >> v2:10.20.193.80:3300/0 conn(0x7f2cdc0c99a0 0x7f2cdc159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce227d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 10 monclient(hunting): get_auth_request con 0x7f2cdc0c99a0 auth_method 0 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:15.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.070+0000 7f2ce1a7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 10 monclient(hunting): handle_auth_done global_id 4758 payload 274 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 1 -- 10.20.193.6:0/3509690853 >> v2:10.20.193.80:3300/0 conn(0x7f2cdc0c99a0 msgr2=0x7f2cdc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 1 --2- 10.20.193.6:0/3509690853 >> v2:10.20.193.80:3300/0 conn(0x7f2cdc0c99a0 0x7f2cdc159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 1 -- 10.20.193.6:0/3509690853 >> v1:10.20.193.80:6789/0 conn(0x7f2cdc15a410 legacy=0x7f2cdc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 10 monclient: _finish_hunting 0 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 1 monclient: found mon.noname-a 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.071+0000 7f2ce227d640 1 -- 10.20.193.6:0/3509690853 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc0c8170 con 0x7f2cdc06ff20 2026-02-22T23:25:15.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.072+0000 7f2ce227d640 1 --2- 10.20.193.6:0/3509690853 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc06ff20 0x7f2cdc070300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009970 tx=0x7f2ccc02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab8d4778e275093 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:15.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 1 -- 10.20.193.6:0/3509690853 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc03c070 con 0x7f2cdc06ff20 2026-02-22T23:25:15.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:15.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: dump: 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: _finish_auth 0 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: _check_auth_tickets 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 1 -- 10.20.193.6:0/3509690853 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ccc02fbc0 con 0x7f2cdc06ff20 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 5 monclient: authenticate success, global_id 4758 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 1 -- 10.20.193.6:0/3509690853 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc037440 con 0x7f2cdc06ff20 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:15.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 4 set_mon_vals no callback set 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce127b640 10 monclient: dump: 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 10 monclient: shutdown 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 -- 10.20.193.6:0/3509690853 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc06ff20 msgr2=0x7f2cdc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3509690853 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc06ff20 0x7f2cdc070300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009970 tx=0x7f2ccc02eef0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 -- 10.20.193.6:0/3509690853 shutdown_connections 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3509690853 >> v2:10.20.193.80:3300/0 conn(0x7f2cdc0c99a0 0x7f2cdc159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3509690853 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc06ff20 0x7f2cdc070300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 -- 10.20.193.6:0/3509690853 >> 10.20.193.6:0/3509690853 conn(0x7f2cdc0cd780 msgr2=0x7f2cdc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 -- 10.20.193.6:0/3509690853 shutdown_connections 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.075+0000 7f2ce4508640 1 -- 10.20.193.6:0/3509690853 wait complete. 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient: build_initial_monmap 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient: monmap: 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 librados: starting msgr at 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 librados: starting objecter 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 Processor -- start 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 -- start start 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 librados: setting wanted keys 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 librados: calling monclient init 2026-02-22T23:25:15.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient: init 2026-02-22T23:25:15.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:15.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:25:15.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2cdc06ff20 0x7f2cdc14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:15.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): picked mon.noname-b con 0x7f2cdc06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:15.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2cdc0c99a0 0x7f2cdc0c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): picked mon.noname-c con 0x7f2cdc0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc15a410 0x7f2cdc1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): picked mon.noname-a con 0x7f2cdc15a410 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc0c8a10 con 0x7f2cdc15a410 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc0c8730 con 0x7f2cdc06ff20 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc150520 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.076+0000 7f2ce4508640 10 monclient(hunting): authenticate will time out at 1189.804964s 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2cdc0c99a0 0x7f2cdc0c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce227d640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2cdc06ff20 0x7f2cdc14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2cdc0c99a0 0x7f2cdc0c06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:57706/0 (socket says 10.20.193.6:57706) 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 1 -- 10.20.193.6:0/3166180015 learned_addr learned my addr 10.20.193.6:0/3166180015 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 10 monclient(hunting): get_auth_request con 0x7f2cdc0c99a0 auth_method 0 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce2a7e640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc15a410 0x7f2cdc1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce227d640 10 monclient(hunting): get_auth_request con 0x7f2cdc06ff20 auth_method 0 2026-02-22T23:25:15.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce227d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce227d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce227d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce2a7e640 10 monclient(hunting): get_auth_request con 0x7f2cdc15a410 auth_method 0 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce2a7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce2a7e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce2a7e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce1a7c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce2a7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.077+0000 7f2ce2a7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.078+0000 7f2ce2a7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.078+0000 7f2ce227d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.078+0000 7f2ce227d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.078+0000 7f2ce227d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 10 monclient(hunting): handle_auth_done global_id 4865 payload 824 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 1 -- 10.20.193.6:0/3166180015 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2cdc06ff20 msgr2=0x7f2cdc14c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2cdc06ff20 0x7f2cdc14c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 1 -- 10.20.193.6:0/3166180015 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc15a410 msgr2=0x7f2cdc1549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc15a410 0x7f2cdc1549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 10 monclient: _finish_hunting 0 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 1 monclient: found mon.noname-c 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.079+0000 7f2ce1a7c640 1 -- 10.20.193.6:0/3166180015 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc0be7a0 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.080+0000 7f2ce1a7c640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2cdc0c99a0 0x7f2cdc0c06f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000ebe0 tx=0x7f2cd000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:15.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.080+0000 7f2cc2ffd640 10 client.?.objecter ms_handle_connect 0x7f2cdc0c99a0 2026-02-22T23:25:15.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.080+0000 7f2cc2ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:15.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 1 -- 10.20.193.6:0/3166180015 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0017070 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:15.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:15.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: dump: 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: _finish_auth 0 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 5 monclient: authenticate success, global_id 4865 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 1 -- 10.20.193.6:0/3166180015 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd000cc40 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 1 -- 10.20.193.6:0/3166180015 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0013470 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 10 monclient: _renew_subs 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce0a7a640 4 set_mon_vals no callback set 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc0bea80 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2cc2ffd640 10 monclient: dump: 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:15.087 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 10 client.4865.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 10 monclient: _renew_subs 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc0bf040 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.081+0000 7f2ce4508640 1 librados: init done 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.082+0000 7f2ce4508640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.082+0000 7f2ce4508640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.082+0000 7f2ce4508640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:15.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.082+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc070300 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.093+0000 7f2cc2ffd640 1 -- 10.20.193.6:0/3166180015 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f2cd0013610 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.094+0000 7f2cc2ffd640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2cac078e00 0x7f2cac07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:15.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.094+0000 7f2ce227d640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2cac078e00 0x7f2cac07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:15.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.094+0000 7f2ce227d640 10 monclient: get_auth_request con 0x7f2cac078e00 auth_method 0 2026-02-22T23:25:15.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.094+0000 7f2ce227d640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2cac078e00 0x7f2cac07b2c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0c5380 tx=0x7f2ccc039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:15.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.094+0000 7f2cc2ffd640 10 client.4865.objecter ms_handle_connect 0x7f2cac078e00 2026-02-22T23:25:15.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.189+0000 7f2cc2ffd640 1 -- 10.20.193.6:0/3166180015 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f2cd005e590 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.189+0000 7f2cc2ffd640 10 client.4865.objecter ms_dispatch2 0x7f2cdc000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:15.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.189+0000 7f2cc2ffd640 3 client.4865.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:15.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.189+0000 7f2cc2ffd640 3 client.4865.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:15.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.189+0000 7f2cc2ffd640 20 client.4865.objecter dump_active .. 0 homeless 2026-02-22T23:25:15.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.194+0000 7f2cc2ffd640 1 -- 10.20.193.6:0/3166180015 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f2cd0062550 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.194+0000 7f2cc2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:15.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.194+0000 7f2cc2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:15.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.294+0000 7f2ce4508640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:15.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.294+0000 7f2ce4508640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:15.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.294+0000 7f2ce4508640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:15.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.294+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2cdc14d5c0 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.297+0000 7f2cc2ffd640 1 -- 10.20.193.6:0/3166180015 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f2cd0066850 con 0x7f2cdc0c99a0 2026-02-22T23:25:15.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.297+0000 7f2cc2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:15.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.297+0000 7f2cc2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:15.300 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:15.300 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:15.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 10 librados: watch_flush enter 2026-02-22T23:25:15.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 10 librados: watch_flush exit 2026-02-22T23:25:15.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 20 client.4865.objecter shutdown clearing up homeless session... 2026-02-22T23:25:15.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 10 client.4865.objecter successfully canceled tick 2026-02-22T23:25:15.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2cac078e00 msgr2=0x7f2cac07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:15.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2cac078e00 0x7f2cac07b2c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0c5380 tx=0x7f2ccc039040 comp rx=0 tx=0).stop 2026-02-22T23:25:15.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 10 monclient: shutdown 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2cdc0c99a0 msgr2=0x7f2cdc0c06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2cdc0c99a0 0x7f2cdc0c06f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000ebe0 tx=0x7f2cd000c520 comp rx=0 tx=0).stop 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 shutdown_connections 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f2cac078e00 0x7f2cac07b2c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f2cdc15a410 0x7f2cdc1549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f2cdc0c99a0 0x7f2cdc0c06f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 --2- 10.20.193.6:0/3166180015 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f2cdc06ff20 0x7f2cdc14c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 >> 10.20.193.6:0/3166180015 conn(0x7f2cdc0cd780 msgr2=0x7f2cdc157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:15.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 shutdown_connections 2026-02-22T23:25:15.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 -- 10.20.193.6:0/3166180015 wait complete. 2026-02-22T23:25:15.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:15.298+0000 7f2ce4508640 1 librados: shutdown 2026-02-22T23:25:18.310 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:18.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.375+0000 7f7592cd3640 10 monclient: get_monmap_and_config 2026-02-22T23:25:18.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.375+0000 7f7592cd3640 10 monclient: build_initial_monmap 2026-02-22T23:25:18.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.375+0000 7f7592cd3640 10 monclient: monmap: 2026-02-22T23:25:18.378 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:18.378 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:18.377228+0000 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:18.377228+0000 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:18.379 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:18.380 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:18.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 1 Processor -- start 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 1 -- start start 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient: init 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient: _add_conns ranks=[4,5,2,1,3,0] 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f758c155f80 0x7f758c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient(hunting): picked mon.noname-b con 0x7f758c155f80 addr v2:10.20.193.40:3300/0 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f758c0ccb40 0x7f758c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient(hunting): picked mon.noname-c con 0x7f758c0ccb40 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:18.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f758c156360 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758c0d0380 con 0x7f758c156360 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f758c06d5b0 con 0x7f758c155f80 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f758c06c9a0 con 0x7f758c0ccb40 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.376+0000 7f7592cd3640 10 monclient(hunting): authenticate will time out at 1193.104753s 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f758c0ccb40 0x7f758c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f758c155f80 0x7f758c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:18.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7591249640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f758c156360 0x7f758c15f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:34888/0 (socket says 10.20.193.6:34888) 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7591249640 1 -- 10.20.193.6:0/1967083181 learned_addr learned my addr 10.20.193.6:0/1967083181 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 10 monclient(hunting): get_auth_request con 0x7f758c155f80 auth_method 0 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 10 monclient(hunting): get_auth_request con 0x7f758c0ccb40 auth_method 0 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:18.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7590a48640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.377+0000 7f7583fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 10 monclient(hunting): handle_auth_done global_id 4780 payload 274 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 1 -- 10.20.193.6:0/1967083181 >> v2:10.20.193.80:3300/0 conn(0x7f758c0ccb40 msgr2=0x7f758c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 1 --2- 10.20.193.6:0/1967083181 >> v2:10.20.193.80:3300/0 conn(0x7f758c0ccb40 0x7f758c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 1 -- 10.20.193.6:0/1967083181 >> v1:10.20.193.80:6789/0 conn(0x7f758c156360 legacy=0x7f758c15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 10 monclient: _finish_hunting 0 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 1 monclient: found mon.noname-b 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:18.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:25:18.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.378+0000 7f7590a48640 1 -- 10.20.193.6:0/1967083181 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c06cc80 con 0x7f758c155f80 2026-02-22T23:25:18.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.379+0000 7f7590a48640 1 --2- 10.20.193.6:0/1967083181 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c15c880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f758c15f380 tx=0x7f757c02ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce43ff866b97647 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 1 -- 10.20.193.6:0/1967083181 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c03c070 con 0x7f758c155f80 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: dump: 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.393 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: _finish_auth 0 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: _check_auth_tickets 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 1 -- 10.20.193.6:0/1967083181 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c02fa60 con 0x7f758c155f80 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 5 monclient: authenticate success, global_id 4780 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:18.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 1 -- 10.20.193.6:0/1967083181 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c02fe30 con 0x7f758c155f80 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 4 set_mon_vals no callback set 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f75837fe640 10 monclient: dump: 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:18.395 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 10 monclient: shutdown 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 -- 10.20.193.6:0/1967083181 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 msgr2=0x7f758c15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 --2- 10.20.193.6:0/1967083181 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c15c880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f758c15f380 tx=0x7f757c02ed50 comp rx=0 tx=0).stop 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 -- 10.20.193.6:0/1967083181 shutdown_connections 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 --2- 10.20.193.6:0/1967083181 >> v2:10.20.193.80:3300/0 conn(0x7f758c0ccb40 0x7f758c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 --2- 10.20.193.6:0/1967083181 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c15c880 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 -- 10.20.193.6:0/1967083181 >> 10.20.193.6:0/1967083181 conn(0x7f758c0cc020 msgr2=0x7f758c0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 -- 10.20.193.6:0/1967083181 shutdown_connections 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 1 -- 10.20.193.6:0/1967083181 wait complete. 2026-02-22T23:25:18.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 10 monclient: build_initial_monmap 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.381+0000 7f7592cd3640 10 monclient: monmap: 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 librados: starting msgr at 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 librados: starting objecter 2026-02-22T23:25:18.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 Processor -- start 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 -- start start 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 librados: setting wanted keys 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 librados: calling monclient init 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient: init 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f758c0ccb40 0x7f758c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): picked mon.noname-a con 0x7f758c0ccb40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): picked mon.noname-b con 0x7f758c155f80 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f758c156360 0x7f758c070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): picked mon.noname-c con 0x7f758c156360 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f758c072eb0 con 0x7f758c0ccb40 2026-02-22T23:25:18.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f758c06d240 con 0x7f758c155f80 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f758c15d860 con 0x7f758c156360 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.382+0000 7f7592cd3640 10 monclient(hunting): authenticate will time out at 1193.110653s 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f758c0ccb40 0x7f758c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f758c156360 0x7f758c070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:41308/0 (socket says 10.20.193.6:41308) 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 1 -- 10.20.193.6:0/2690206927 learned_addr learned my addr 10.20.193.6:0/2690206927 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 10 monclient(hunting): get_auth_request con 0x7f758c155f80 auth_method 0 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:18.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 10 monclient(hunting): get_auth_request con 0x7f758c0ccb40 auth_method 0 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 10 monclient(hunting): get_auth_request con 0x7f758c156360 auth_method 0 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7583fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:18.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7591249640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.383+0000 7f7590a48640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 10 monclient(hunting): handle_auth_done global_id 4783 payload 824 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 1 -- 10.20.193.6:0/2690206927 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f758c156360 msgr2=0x7f758c070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f758c156360 0x7f758c070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 1 -- 10.20.193.6:0/2690206927 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f758c0ccb40 msgr2=0x7f758c1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f758c0ccb40 0x7f758c1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 1 monclient: found mon.noname-b 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:25:18.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.384+0000 7f7583fff640 1 -- 10.20.193.6:0/2690206927 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c06f4f0 con 0x7f758c155f80 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.385+0000 7f7583fff640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c158d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f757400ebe0 tx=0x7f757400c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.385+0000 7f75817fa640 10 client.?.objecter ms_handle_connect 0x7f758c155f80 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.385+0000 7f75817fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 1 -- 10.20.193.6:0/2690206927 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574017070 con 0x7f758c155f80 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: dump: 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:18.402 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: _finish_auth 0 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: _check_auth_tickets 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 5 monclient: authenticate success, global_id 4783 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 1 -- 10.20.193.6:0/2690206927 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757400cc40 con 0x7f758c155f80 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 1 -- 10.20.193.6:0/2690206927 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574013470 con 0x7f758c155f80 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 10 monclient: _renew_subs 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c06f7d0 con 0x7f758c155f80 2026-02-22T23:25:18.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7582ffd640 4 set_mon_vals no callback set 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f75817fa640 10 monclient: dump: 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 10 client.4783.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:18.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 10 monclient: _renew_subs 2026-02-22T23:25:18.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:18.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c06fd90 con 0x7f758c155f80 2026-02-22T23:25:18.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.387+0000 7f7592cd3640 1 librados: init done 2026-02-22T23:25:18.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.388+0000 7f7592cd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:18.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.388+0000 7f7592cd3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:18.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.388+0000 7f7592cd3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:18.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.388+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758c15dbf0 con 0x7f758c155f80 2026-02-22T23:25:18.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.494+0000 7f75817fa640 1 -- 10.20.193.6:0/2690206927 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7574013610 con 0x7f758c155f80 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.494+0000 7f75817fa640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f755c078e80 0x7f755c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f7590a48640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f755c078e80 0x7f755c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f7590a48640 10 monclient: get_auth_request con 0x7f755c078e80 auth_method 0 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f75817fa640 1 -- 10.20.193.6:0/2690206927 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f757405e4a0 con 0x7f758c155f80 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f75817fa640 10 client.4783.objecter ms_dispatch2 0x7f758c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f75817fa640 3 client.4783.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f75817fa640 3 client.4783.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:18.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f75817fa640 20 client.4783.objecter dump_active .. 0 homeless 2026-02-22T23:25:18.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f75817fa640 10 client.4783.objecter ms_handle_connect 0x7f755c078e80 2026-02-22T23:25:18.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.495+0000 7f7590a48640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f755c078e80 0x7f755c07b340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f757c039000 tx=0x7f757c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:18.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.501+0000 7f75817fa640 1 -- 10.20.193.6:0/2690206927 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7574062460 con 0x7f758c155f80 2026-02-22T23:25:18.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.501+0000 7f75817fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:18.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.501+0000 7f75817fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:18.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.601+0000 7f7592cd3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:18.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.601+0000 7f7592cd3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:18.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.601+0000 7f7592cd3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:18.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.601+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f758c072bd0 con 0x7f758c155f80 2026-02-22T23:25:18.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f75817fa640 1 -- 10.20.193.6:0/2690206927 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7574066760 con 0x7f758c155f80 2026-02-22T23:25:18.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f75817fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:18.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f75817fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:18.606 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:18.607 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:18.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 10 librados: watch_flush enter 2026-02-22T23:25:18.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 10 librados: watch_flush exit 2026-02-22T23:25:18.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 20 client.4783.objecter shutdown clearing up homeless session... 2026-02-22T23:25:18.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 10 client.4783.objecter successfully canceled tick 2026-02-22T23:25:18.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f755c078e80 msgr2=0x7f755c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f755c078e80 0x7f755c07b340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f757c039000 tx=0x7f757c033000 comp rx=0 tx=0).stop 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 10 monclient: shutdown 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 msgr2=0x7f758c158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c158d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f757400ebe0 tx=0x7f757400c520 comp rx=0 tx=0).stop 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 shutdown_connections 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f755c078e80 0x7f755c07b340 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f758c156360 0x7f758c070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f758c155f80 0x7f758c158d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 --2- 10.20.193.6:0/2690206927 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f758c0ccb40 0x7f758c1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:18.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 >> 10.20.193.6:0/2690206927 conn(0x7f758c0cc020 msgr2=0x7f758c0cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:18.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 shutdown_connections 2026-02-22T23:25:18.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 -- 10.20.193.6:0/2690206927 wait complete. 2026-02-22T23:25:18.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:18.604+0000 7f7592cd3640 1 librados: shutdown 2026-02-22T23:25:21.617 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.682+0000 7f71b6424640 10 monclient: get_monmap_and_config 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.682+0000 7f71b6424640 10 monclient: build_initial_monmap 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.682+0000 7f71b6424640 10 monclient: monmap: 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:21.684494+0000 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:21.684494+0000 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:21.685 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:21.686 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:21.686 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:21.686 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 1 Processor -- start 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 1 -- start start 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient: init 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient: _add_conns ranks=[3,2,5,1,4,0] 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f71b006ff20 0x7f71b0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient(hunting): picked mon.noname-a con 0x7f71b006ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f71b00c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f71b015a5e0 0x7f71b014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient(hunting): picked mon.noname-c con 0x7f71b015a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71b00c7e90 con 0x7f71b00c99a0 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f71b0159a40 con 0x7f71b006ff20 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f71b00c8170 con 0x7f71b015a5e0 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.683+0000 7f71b6424640 10 monclient(hunting): authenticate will time out at 1196.411826s 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f71b006ff20 0x7f71b0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71b499a640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f71b015a5e0 0x7f71b014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:21.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f71b006ff20 0x7f71b0070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:35716/0 (socket says 10.20.193.6:35716) 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 1 -- 10.20.193.6:0/1440717087 learned_addr learned my addr 10.20.193.6:0/1440717087 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 10 monclient(hunting): get_auth_request con 0x7f71b006ff20 auth_method 0 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71b499a640 10 monclient(hunting): get_auth_request con 0x7f71b015a5e0 auth_method 0 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71b499a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71b499a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:21.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.684+0000 7f71b499a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71b499a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71b499a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71b499a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 10 monclient(hunting): handle_auth_done global_id 4767 payload 274 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 1 -- 10.20.193.6:0/1440717087 >> v2:10.20.193.80:3300/0 conn(0x7f71b015a5e0 msgr2=0x7f71b014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 1 --2- 10.20.193.6:0/1440717087 >> v2:10.20.193.80:3300/0 conn(0x7f71b015a5e0 0x7f71b014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 1 -- 10.20.193.6:0/1440717087 >> v1:10.20.193.80:6789/0 conn(0x7f71b00c99a0 legacy=0x7f71b0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 1 monclient: found mon.noname-a 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.685+0000 7f71affff640 1 -- 10.20.193.6:0/1440717087 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b00c8450 con 0x7f71b006ff20 2026-02-22T23:25:21.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.686+0000 7f71affff640 1 --2- 10.20.193.6:0/1440717087 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b006ff20 0x7f71b0070300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f719c009970 tx=0x7f719c02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aef8f36df93ad7 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 1 -- 10.20.193.6:0/1440717087 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c03c070 con 0x7f71b006ff20 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: dump: 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: _finish_auth 0 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 1 -- 10.20.193.6:0/1440717087 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719c02fbc0 con 0x7f71b006ff20 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 5 monclient: authenticate success, global_id 4767 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 1 -- 10.20.193.6:0/1440717087 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c037440 con 0x7f71b006ff20 2026-02-22T23:25:21.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71aeffd640 10 monclient: dump: 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 4 set_mon_vals no callback set 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 10 monclient: shutdown 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 -- 10.20.193.6:0/1440717087 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b006ff20 msgr2=0x7f71b0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 --2- 10.20.193.6:0/1440717087 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b006ff20 0x7f71b0070300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f719c009970 tx=0x7f719c02eef0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 -- 10.20.193.6:0/1440717087 shutdown_connections 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 --2- 10.20.193.6:0/1440717087 >> v2:10.20.193.80:3300/0 conn(0x7f71b015a5e0 0x7f71b014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 --2- 10.20.193.6:0/1440717087 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b006ff20 0x7f71b0070300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 -- 10.20.193.6:0/1440717087 >> 10.20.193.6:0/1440717087 conn(0x7f71b00cd780 msgr2=0x7f71b00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 -- 10.20.193.6:0/1440717087 shutdown_connections 2026-02-22T23:25:21.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.688+0000 7f71b6424640 1 -- 10.20.193.6:0/1440717087 wait complete. 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 10 monclient: build_initial_monmap 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 10 monclient: monmap: 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 1 librados: starting msgr at 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 1 librados: starting objecter 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 1 Processor -- start 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 1 -- start start 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 1 librados: setting wanted keys 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 1 librados: calling monclient init 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.689+0000 7f71b6424640 10 monclient: init 2026-02-22T23:25:21.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f71b006ff20 0x7f71b01f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): picked mon.noname-b con 0x7f71b006ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b00c99a0 0x7f71b01f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): picked mon.noname-a con 0x7f71b00c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f71b015a5e0 0x7f71b0072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): picked mon.noname-c con 0x7f71b015a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f71b0150350 con 0x7f71b00c99a0 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f71b00c8a10 con 0x7f71b006ff20 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f71b01504d0 con 0x7f71b015a5e0 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b6424640 10 monclient(hunting): authenticate will time out at 1196.418353s 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b00c99a0 0x7f71b01f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b00c99a0 0x7f71b01f3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:35724/0 (socket says 10.20.193.6:35724) 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 1 -- 10.20.193.6:0/2964482821 learned_addr learned my addr 10.20.193.6:0/2964482821 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 10 monclient(hunting): get_auth_request con 0x7f71b00c99a0 auth_method 0 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b499a640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f71b015a5e0 0x7f71b0072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71af7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:21.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b499a640 10 monclient(hunting): get_auth_request con 0x7f71b015a5e0 auth_method 0 2026-02-22T23:25:21.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b499a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:21.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b499a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:21.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.690+0000 7f71b499a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:21.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.691+0000 7f71b499a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:21.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.691+0000 7f71b499a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:21.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.691+0000 7f71b499a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 10 monclient(hunting): handle_auth_done global_id 4770 payload 824 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 1 -- 10.20.193.6:0/2964482821 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f71b015a5e0 msgr2=0x7f71b0072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f71b015a5e0 0x7f71b0072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 1 -- 10.20.193.6:0/2964482821 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f71b006ff20 msgr2=0x7f71b01f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f71b006ff20 0x7f71b01f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 1 monclient: found mon.noname-a 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.692+0000 7f71af7fe640 1 -- 10.20.193.6:0/2964482821 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b00c10d0 con 0x7f71b00c99a0 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.693+0000 7f71af7fe640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b00c99a0 0x7f71b01f3730 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f71a000c580 tx=0x7f71a000ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.693+0000 7f71acff9640 10 client.?.objecter ms_handle_connect 0x7f71b00c99a0 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.693+0000 7f71acff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 1 -- 10.20.193.6:0/2964482821 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a0017070 con 0x7f71b00c99a0 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:21.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: dump: 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: _finish_auth 0 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: _check_auth_tickets 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 1 -- 10.20.193.6:0/2964482821 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a00052d0 con 0x7f71b00c99a0 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 5 monclient: authenticate success, global_id 4770 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 10 monclient: _renew_subs 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b01f3c70 con 0x7f71b00c99a0 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 1 -- 10.20.193.6:0/2964482821 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a00036b0 con 0x7f71b00c99a0 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71acff9640 10 monclient: dump: 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:21.698 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71ae7fc640 4 set_mon_vals no callback set 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 10 client.4770.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 10 monclient: _renew_subs 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b01f4220 con 0x7f71b00c99a0 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.694+0000 7f71b6424640 1 librados: init done 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.695+0000 7f71b6424640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.695+0000 7f71b6424640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.695+0000 7f71b6424640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:21.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.695+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b0150860 con 0x7f71b00c99a0 2026-02-22T23:25:21.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.705+0000 7f71acff9640 1 -- 10.20.193.6:0/2964482821 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f71a0003850 con 0x7f71b00c99a0 2026-02-22T23:25:21.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.706+0000 7f71acff9640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7180078e00 0x7f718007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:21.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.706+0000 7f71affff640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7180078e00 0x7f718007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:21.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.706+0000 7f71affff640 10 monclient: get_auth_request con 0x7f7180078e00 auth_method 0 2026-02-22T23:25:21.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.707+0000 7f71acff9640 1 -- 10.20.193.6:0/2964482821 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f71a005d400 con 0x7f71b00c99a0 2026-02-22T23:25:21.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.707+0000 7f71acff9640 10 client.4770.objecter ms_dispatch2 0x7f71b0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:21.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.707+0000 7f71acff9640 3 client.4770.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:21.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.707+0000 7f71acff9640 3 client.4770.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:21.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.707+0000 7f71acff9640 20 client.4770.objecter dump_active .. 0 homeless 2026-02-22T23:25:21.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.707+0000 7f71acff9640 10 client.4770.objecter ms_handle_connect 0x7f7180078e00 2026-02-22T23:25:21.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.707+0000 7f71affff640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7180078e00 0x7f718007b2c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71b00c5350 tx=0x7f719c031000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:21.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.804+0000 7f71acff9640 1 -- 10.20.193.6:0/2964482821 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f71a0021020 con 0x7f71b00c99a0 2026-02-22T23:25:21.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.804+0000 7f71acff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:21.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.804+0000 7f71acff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:21.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.904+0000 7f71b6424640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:21.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.904+0000 7f71b6424640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:21.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.904+0000 7f71b6424640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:21.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.904+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f71b014f510 con 0x7f71b00c99a0 2026-02-22T23:25:21.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.909+0000 7f71acff9640 1 -- 10.20.193.6:0/2964482821 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f71a00613c0 con 0x7f71b00c99a0 2026-02-22T23:25:21.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.909+0000 7f71acff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:21.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.909+0000 7f71acff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:21.911 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:21.911 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 10 librados: watch_flush enter 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 10 librados: watch_flush exit 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 20 client.4770.objecter shutdown clearing up homeless session... 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 10 client.4770.objecter successfully canceled tick 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7180078e00 msgr2=0x7f718007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7180078e00 0x7f718007b2c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71b00c5350 tx=0x7f719c031000 comp rx=0 tx=0).stop 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 10 monclient: shutdown 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b00c99a0 msgr2=0x7f71b01f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:21.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b00c99a0 0x7f71b01f3730 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f71a000c580 tx=0x7f71a000ca50 comp rx=0 tx=0).stop 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 shutdown_connections 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7180078e00 0x7f718007b2c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f71b015a5e0 0x7f71b0072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f71b00c99a0 0x7f71b01f3730 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 --2- 10.20.193.6:0/2964482821 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f71b006ff20 0x7f71b01f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 >> 10.20.193.6:0/2964482821 conn(0x7f71b00cd780 msgr2=0x7f71b014e580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 shutdown_connections 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 -- 10.20.193.6:0/2964482821 wait complete. 2026-02-22T23:25:21.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:21.910+0000 7f71b6424640 1 librados: shutdown 2026-02-22T23:25:24.923 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:24.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.991+0000 7fb00a874640 10 monclient: get_monmap_and_config 2026-02-22T23:25:24.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.991+0000 7fb00a874640 10 monclient: build_initial_monmap 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.991+0000 7fb00a874640 10 monclient: monmap: 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:24.992701+0000 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:24.992701+0000 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:24.994 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:24.995 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:24.995 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:24.995 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:24.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.991+0000 7fb00a874640 1 Processor -- start 2026-02-22T23:25:24.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.991+0000 7fb00a874640 1 -- start start 2026-02-22T23:25:24.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.991+0000 7fb00a874640 10 monclient: init 2026-02-22T23:25:24.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:24.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient: _add_conns ranks=[3,2,4,1,5,0] 2026-02-22T23:25:24.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fb0041527f0 0x7fb004073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient(hunting): picked mon.noname-a con 0x7fb0041527f0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb00415d2c0 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fb00415d730 0x7fb0040ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient(hunting): picked mon.noname-b con 0x7fb00415d730 addr v2:10.20.193.40:3300/0 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb004073c20 con 0x7fb00415d2c0 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb004074b10 con 0x7fb0041527f0 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb004073f00 con 0x7fb00415d730 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb00a874640 10 monclient(hunting): authenticate will time out at 1199.720245s 2026-02-22T23:25:24.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb003fff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fb0041527f0 0x7fb004073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb0037fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fb00415d2c0 0x7fb00415f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:34906/0 (socket says 10.20.193.6:34906) 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb0037fe640 1 -- 10.20.193.6:0/2053481471 learned_addr learned my addr 10.20.193.6:0/2053481471 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb008dea640 1 --2- 10.20.193.6:0/2053481471 >> v2:10.20.193.40:3300/0 conn(0x7fb00415d730 0x7fb0040ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb003fff640 10 monclient(hunting): get_auth_request con 0x7fb0041527f0 auth_method 0 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb003fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb003fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.992+0000 7fb003fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb008dea640 10 monclient(hunting): get_auth_request con 0x7fb00415d730 auth_method 0 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb008dea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb008dea640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb008dea640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb003fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:25.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb003fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb003fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb008dea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb008dea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.993+0000 7fb008dea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb002ffd640 1 -- 10.20.193.6:0/2053481471 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3231078962 0 0) 0x7fb004073c20 con 0x7fb00415d2c0 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb002ffd640 10 monclient(hunting): _negotiate 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb002ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb002ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb002ffd640 10 monclient(hunting): my global_id is 4886 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb002ffd640 1 -- 10.20.193.6:0/2053481471 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafe8003620 con 0x7fb00415d2c0 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 10 monclient(hunting): handle_auth_done global_id 4773 payload 274 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 1 -- 10.20.193.6:0/2053481471 >> v2:10.20.193.40:3300/0 conn(0x7fb00415d730 msgr2=0x7fb0040ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:25.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 1 --2- 10.20.193.6:0/2053481471 >> v2:10.20.193.40:3300/0 conn(0x7fb00415d730 0x7fb0040ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 1 -- 10.20.193.6:0/2053481471 >> v1:10.20.193.80:6789/0 conn(0x7fb00415d2c0 legacy=0x7fb00415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:25.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:25.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 1 monclient: found mon.noname-a 2026-02-22T23:25:25.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:25.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:25.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.994+0000 7fb003fff640 1 -- 10.20.193.6:0/2053481471 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0040741e0 con 0x7fb0041527f0 2026-02-22T23:25:25.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.995+0000 7fb003fff640 1 --2- 10.20.193.6:0/2053481471 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb0041527f0 0x7fb004073530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb00406cdf0 tx=0x7faff002ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b53f71ebdbb5e2e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:25.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 1 -- 10.20.193.6:0/2053481471 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff003c070 con 0x7fb0041527f0 2026-02-22T23:25:25.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:25.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:25.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: dump: 2026-02-22T23:25:25.008 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: _finish_auth 0 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:25.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 1 -- 10.20.193.6:0/2053481471 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff002faf0 con 0x7fb0041527f0 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 1 -- 10.20.193.6:0/2053481471 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0037440 con 0x7fb0041527f0 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 5 monclient: authenticate success, global_id 4773 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 4 set_mon_vals no callback set 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb002ffd640 10 monclient: dump: 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:25.010 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 10 monclient: shutdown 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 -- 10.20.193.6:0/2053481471 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb0041527f0 msgr2=0x7fb004073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 --2- 10.20.193.6:0/2053481471 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb0041527f0 0x7fb004073530 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb00406cdf0 tx=0x7faff002ede0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 -- 10.20.193.6:0/2053481471 shutdown_connections 2026-02-22T23:25:25.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 --2- 10.20.193.6:0/2053481471 >> v2:10.20.193.40:3300/0 conn(0x7fb00415d730 0x7fb0040ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 --2- 10.20.193.6:0/2053481471 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb0041527f0 0x7fb004073530 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 -- 10.20.193.6:0/2053481471 >> 10.20.193.6:0/2053481471 conn(0x7fb0040ce730 msgr2=0x7fb0040ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 -- 10.20.193.6:0/2053481471 shutdown_connections 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.997+0000 7fb00a874640 1 -- 10.20.193.6:0/2053481471 wait complete. 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient: build_initial_monmap 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient: monmap: 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:25.012 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 librados: starting msgr at 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 librados: starting objecter 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 Processor -- start 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 -- start start 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 librados: setting wanted keys 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 librados: calling monclient init 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient: init 2026-02-22T23:25:25.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb0041527f0 0x7fb0041f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): picked mon.noname-b con 0x7fb0041527f0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb00415d2c0 0x7fb0041f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): picked mon.noname-c con 0x7fb00415d2c0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb00415d730 0x7fb00414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): picked mon.noname-a con 0x7fb00415d730 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb004074fa0 con 0x7fb00415d730 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb0040747a0 con 0x7fb0041527f0 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:25.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb004075120 con 0x7fb00415d2c0 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.998+0000 7fb00a874640 10 monclient(hunting): authenticate will time out at 1199.726801s 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb00415d730 0x7fb00414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb0041527f0 0x7fb0041f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb00415d730 0x7fb00414ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:35734/0 (socket says 10.20.193.6:35734) 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 1 -- 10.20.193.6:0/2110662574 learned_addr learned my addr 10.20.193.6:0/2110662574 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 10 monclient(hunting): get_auth_request con 0x7fb00415d730 auth_method 0 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 10 monclient(hunting): get_auth_request con 0x7fb0041527f0 auth_method 0 2026-02-22T23:25:25.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb00415d2c0 0x7fb0041f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb008dea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 10 monclient(hunting): get_auth_request con 0x7fb00415d2c0 auth_method 0 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:25.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb003fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:24.999+0000 7fb0037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 10 monclient(hunting): handle_auth_done global_id 4789 payload 824 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 1 -- 10.20.193.6:0/2110662574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb00415d2c0 msgr2=0x7fb0041f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb00415d2c0 0x7fb0041f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 1 -- 10.20.193.6:0/2110662574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb00415d730 msgr2=0x7fb00414ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb00415d730 0x7fb00414ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 1 monclient: found mon.noname-b 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:25.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.000+0000 7fb003fff640 1 -- 10.20.193.6:0/2110662574 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0041f4120 con 0x7fb0041527f0 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.002+0000 7fb003fff640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb0041527f0 0x7fb0041f6d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faff0036000 tx=0x7faff00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.002+0000 7fb000ff9640 10 client.?.objecter ms_handle_connect 0x7fb0041527f0 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.002+0000 7fb000ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 1 -- 10.20.193.6:0/2110662574 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff003c040 con 0x7fb0041527f0 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: dump: 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.018 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: _finish_auth 0 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 5 monclient: authenticate success, global_id 4789 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 monclient: _renew_subs 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:25.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0041f6850 con 0x7fb0041527f0 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 client.4789.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 monclient: _renew_subs 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0041f35a0 con 0x7fb0041527f0 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 1 librados: init done 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 1 -- 10.20.193.6:0/2110662574 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff002f9b0 con 0x7fb0041527f0 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 1 -- 10.20.193.6:0/2110662574 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0042650 con 0x7fb0041527f0 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb000ff9640 10 monclient: dump: 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:25.020 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb0027fc640 4 set_mon_vals no callback set 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:25.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.003+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb004152bd0 con 0x7fb0041527f0 2026-02-22T23:25:25.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.014+0000 7fb000ff9640 1 -- 10.20.193.6:0/2110662574 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7faff00427f0 con 0x7fb0041527f0 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.015+0000 7fb000ff9640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafd4078e00 0x7fafd407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.015+0000 7fb0037fe640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafd4078e00 0x7fafd407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.015+0000 7fb0037fe640 10 monclient: get_auth_request con 0x7fafd4078e00 auth_method 0 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.016+0000 7fb0037fe640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafd4078e00 0x7fafd407b2c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faff4007b60 tx=0x7faff4007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.016+0000 7fb000ff9640 10 client.4789.objecter ms_handle_connect 0x7fafd4078e00 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.016+0000 7fb000ff9640 1 -- 10.20.193.6:0/2110662574 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7faff0083520 con 0x7fb0041527f0 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.016+0000 7fb000ff9640 10 client.4789.objecter ms_dispatch2 0x7fb004000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.016+0000 7fb000ff9640 3 client.4789.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.016+0000 7fb000ff9640 3 client.4789.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:25.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.017+0000 7fb000ff9640 20 client.4789.objecter dump_active .. 0 homeless 2026-02-22T23:25:25.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.117+0000 7fb000ff9640 1 -- 10.20.193.6:0/2110662574 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7faff00874e0 con 0x7fb0041527f0 2026-02-22T23:25:25.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.117+0000 7fb000ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:25.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.117+0000 7fb000ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:25.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.218+0000 7fb00a874640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:25.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.218+0000 7fb00a874640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:25.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.218+0000 7fb00a874640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:25.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.218+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb0041f3880 con 0x7fb0041527f0 2026-02-22T23:25:25.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.234+0000 7fb000ff9640 1 -- 10.20.193.6:0/2110662574 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7faff008b7e0 con 0x7fb0041527f0 2026-02-22T23:25:25.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.234+0000 7fb000ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:25.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.234+0000 7fb000ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:25.237 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:25.237 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:25.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 10 librados: watch_flush enter 2026-02-22T23:25:25.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 10 librados: watch_flush exit 2026-02-22T23:25:25.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 20 client.4789.objecter shutdown clearing up homeless session... 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 10 client.4789.objecter successfully canceled tick 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafd4078e00 msgr2=0x7fafd407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafd4078e00 0x7fafd407b2c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faff4007b60 tx=0x7faff4007ab0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 10 monclient: shutdown 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb0041527f0 msgr2=0x7fb0041f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb0041527f0 0x7fb0041f6d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faff0036000 tx=0x7faff00047c0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 shutdown_connections 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fafd4078e00 0x7fafd407b2c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb00415d730 0x7fb00414ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb00415d2c0 0x7fb0041f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 --2- 10.20.193.6:0/2110662574 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb0041527f0 0x7fb0041f6d10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:25.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 >> 10.20.193.6:0/2110662574 conn(0x7fb0040ce730 msgr2=0x7fb004151630 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:25.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 shutdown_connections 2026-02-22T23:25:25.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 -- 10.20.193.6:0/2110662574 wait complete. 2026-02-22T23:25:25.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:25.236+0000 7fb00a874640 1 librados: shutdown 2026-02-22T23:25:28.248 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.315+0000 7f1d49b86640 10 monclient: get_monmap_and_config 2026-02-22T23:25:28.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.315+0000 7f1d49b86640 10 monclient: build_initial_monmap 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.315+0000 7f1d49b86640 10 monclient: monmap: 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:28.317293+0000 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:28.317293+0000 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:28.321 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 1 Processor -- start 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 1 -- start start 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient: init 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient: _add_conns ranks=[2,1,3,0,4,5] 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1d4406ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:28.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1d440c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1d4415a5e0 0x7f1d44070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient(hunting): picked mon.noname-a con 0x7f1d4415a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440c7e90 con 0x7f1d440c99a0 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440c8170 con 0x7f1d4406ff20 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1d44159a40 con 0x7f1d4415a5e0 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.316+0000 7f1d49b86640 10 monclient(hunting): authenticate will time out at 1203.044657s 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1d4415a5e0 0x7f1d44070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1d4415a5e0 0x7f1d44070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54148/0 (socket says 10.20.193.6:54148) 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 1 -- 10.20.193.6:0/4011004176 learned_addr learned my addr 10.20.193.6:0/4011004176 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:28.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 10 monclient(hunting): get_auth_request con 0x7f1d4415a5e0 auth_method 0 2026-02-22T23:25:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:28.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.317+0000 7f1d43fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 10 monclient(hunting): handle_auth_done global_id 4779 payload 274 2026-02-22T23:25:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 1 -- 10.20.193.6:0/4011004176 >> v1:10.20.193.80:6789/0 conn(0x7f1d4406ff20 legacy=0x7f1d44159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:28.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 1 -- 10.20.193.6:0/4011004176 >> v1:10.20.193.40:6789/0 conn(0x7f1d440c99a0 legacy=0x7f1d4414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 1 monclient: found mon.noname-a 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.318+0000 7f1d43fff640 1 -- 10.20.193.6:0/4011004176 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440c8450 con 0x7f1d4415a5e0 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.319+0000 7f1d43fff640 1 --2- 10.20.193.6:0/4011004176 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4415a5e0 0x7f1d44070300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009e80 tx=0x7f1d3802f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4990f1aaef91ae7d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 1 -- 10.20.193.6:0/4011004176 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3802fcf0 con 0x7f1d4415a5e0 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:28.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: dump: 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:28.331 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: _finish_auth 0 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: _check_auth_tickets 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 1 -- 10.20.193.6:0/4011004176 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d380373f0 con 0x7f1d4415a5e0 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 5 monclient: authenticate success, global_id 4779 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 1 -- 10.20.193.6:0/4011004176 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d380377c0 con 0x7f1d4415a5e0 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 4 set_mon_vals no callback set 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:28.332 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d427fc640 10 monclient: dump: 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:28.333 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 10 monclient: shutdown 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 1 -- 10.20.193.6:0/4011004176 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4415a5e0 msgr2=0x7f1d44070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 1 --2- 10.20.193.6:0/4011004176 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4415a5e0 0x7f1d44070300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009e80 tx=0x7f1d3802f2d0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 1 -- 10.20.193.6:0/4011004176 shutdown_connections 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 1 --2- 10.20.193.6:0/4011004176 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4415a5e0 0x7f1d44070300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 1 -- 10.20.193.6:0/4011004176 >> 10.20.193.6:0/4011004176 conn(0x7f1d440cd780 msgr2=0x7f1d440cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 1 -- 10.20.193.6:0/4011004176 shutdown_connections 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.321+0000 7f1d49b86640 1 -- 10.20.193.6:0/4011004176 wait complete. 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.322+0000 7f1d49b86640 10 monclient: build_initial_monmap 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.322+0000 7f1d49b86640 10 monclient: monmap: 2026-02-22T23:25:28.334 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:28.335 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.322+0000 7f1d49b86640 1 librados: starting msgr at 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.322+0000 7f1d49b86640 1 librados: starting objecter 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.322+0000 7f1d49b86640 1 Processor -- start 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 -- start start 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 librados: setting wanted keys 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 librados: calling monclient init 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient: init 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4406ff20 0x7f1d4406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): picked mon.noname-a con 0x7f1d4406ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d440c99a0 0x7f1d440c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:28.336 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): picked mon.noname-b con 0x7f1d440c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d4415a5e0 0x7f1d440c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): picked mon.noname-c con 0x7f1d4415a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44150520 con 0x7f1d4406ff20 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440c8a10 con 0x7f1d440c99a0 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441506a0 con 0x7f1d4415a5e0 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d49b86640 10 monclient(hunting): authenticate will time out at 1203.051524s 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d437fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4406ff20 0x7f1d4406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:28.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d437fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4406ff20 0x7f1d4406ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54162/0 (socket says 10.20.193.6:54162) 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d437fe640 1 -- 10.20.193.6:0/2372294301 learned_addr learned my addr 10.20.193.6:0/2372294301 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d437fe640 10 monclient(hunting): get_auth_request con 0x7f1d4406ff20 auth_method 0 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d437fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d437fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.323+0000 7f1d437fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.324+0000 7f1d437fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.324+0000 7f1d437fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.324+0000 7f1d437fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 10 monclient(hunting): handle_auth_done global_id 4782 payload 824 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 1 -- 10.20.193.6:0/2372294301 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d4415a5e0 msgr2=0x7f1d440c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d4415a5e0 0x7f1d440c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 1 -- 10.20.193.6:0/2372294301 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d440c99a0 msgr2=0x7f1d440c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d440c99a0 0x7f1d440c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 10 monclient: _finish_hunting 0 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 1 monclient: found mon.noname-a 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.325+0000 7f1d437fe640 1 -- 10.20.193.6:0/2372294301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d441e68f0 con 0x7f1d4406ff20 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.326+0000 7f1d437fe640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4406ff20 0x7f1d4406ef50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b4d0 tx=0x7f1d3000b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.326+0000 7f1d1ffff640 10 client.?.objecter ms_handle_connect 0x7f1d4406ff20 2026-02-22T23:25:28.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.326+0000 7f1d1ffff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:28.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 1 -- 10.20.193.6:0/2372294301 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30004480 con 0x7f1d4406ff20 2026-02-22T23:25:28.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:28.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:28.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: dump: 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:28.342 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: _finish_auth 0 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: _check_auth_tickets 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 1 -- 10.20.193.6:0/2372294301 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d30015070 con 0x7f1d4406ff20 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 1 -- 10.20.193.6:0/2372294301 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30002920 con 0x7f1d4406ff20 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 5 monclient: authenticate success, global_id 4782 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 monclient: _renew_subs 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:28.343 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d441e6b70 con 0x7f1d4406ff20 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d41ffb640 4 set_mon_vals no callback set 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d1ffff640 10 monclient: dump: 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:28.344 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 client.4782.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 monclient: _renew_subs 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441e7130 con 0x7f1d4406ff20 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 1 librados: init done 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:28.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.328+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d44070300 con 0x7f1d4406ff20 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.340+0000 7f1d1ffff640 1 -- 10.20.193.6:0/2372294301 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1d30010480 con 0x7f1d4406ff20 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.340+0000 7f1d1ffff640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d14078e50 0x7f1d1407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.340+0000 7f1d42ffd640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d14078e50 0x7f1d1407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.340+0000 7f1d42ffd640 10 monclient: get_auth_request con 0x7f1d14078e50 auth_method 0 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.341+0000 7f1d42ffd640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d14078e50 0x7f1d1407b310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d34006fd0 tx=0x7f1d34006d40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.341+0000 7f1d1ffff640 10 client.4782.objecter ms_handle_connect 0x7f1d14078e50 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.342+0000 7f1d1ffff640 1 -- 10.20.193.6:0/2372294301 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f1d3005e200 con 0x7f1d4406ff20 2026-02-22T23:25:28.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.342+0000 7f1d1ffff640 10 client.4782.objecter ms_dispatch2 0x7f1d44000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:28.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.342+0000 7f1d1ffff640 3 client.4782.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:28.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.342+0000 7f1d1ffff640 3 client.4782.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:28.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.342+0000 7f1d1ffff640 20 client.4782.objecter dump_active .. 0 homeless 2026-02-22T23:25:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.436+0000 7f1d1ffff640 1 -- 10.20.193.6:0/2372294301 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1d441e68f0 con 0x7f1d4406ff20 2026-02-22T23:25:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.436+0000 7f1d1ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:28.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.436+0000 7f1d1ffff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:28.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.538+0000 7f1d49b86640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:28.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.538+0000 7f1d49b86640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:28.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.538+0000 7f1d49b86640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:28.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.538+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1d44150170 con 0x7f1d4406ff20 2026-02-22T23:25:28.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.540+0000 7f1d1ffff640 1 -- 10.20.193.6:0/2372294301 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1d3000d360 con 0x7f1d4406ff20 2026-02-22T23:25:28.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.540+0000 7f1d1ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:28.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.540+0000 7f1d1ffff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:28.543 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:28.543 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:28.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 10 librados: watch_flush enter 2026-02-22T23:25:28.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 10 librados: watch_flush exit 2026-02-22T23:25:28.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 20 client.4782.objecter shutdown clearing up homeless session... 2026-02-22T23:25:28.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 10 client.4782.objecter successfully canceled tick 2026-02-22T23:25:28.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d14078e50 msgr2=0x7f1d1407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d14078e50 0x7f1d1407b310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d34006fd0 tx=0x7f1d34006d40 comp rx=0 tx=0).stop 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 10 monclient: shutdown 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4406ff20 msgr2=0x7f1d4406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4406ff20 0x7f1d4406ef50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b4d0 tx=0x7f1d3000b9a0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 shutdown_connections 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1d14078e50 0x7f1d1407b310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1d4415a5e0 0x7f1d440c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1d440c99a0 0x7f1d440c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 --2- 10.20.193.6:0/2372294301 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1d4406ff20 0x7f1d4406ef50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:28.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 >> 10.20.193.6:0/2372294301 conn(0x7f1d440cd780 msgr2=0x7f1d440c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:28.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 shutdown_connections 2026-02-22T23:25:28.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 -- 10.20.193.6:0/2372294301 wait complete. 2026-02-22T23:25:28.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:28.541+0000 7f1d49b86640 1 librados: shutdown 2026-02-22T23:25:31.554 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.621+0000 7f752ca1a640 10 monclient: get_monmap_and_config 2026-02-22T23:25:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.621+0000 7f752ca1a640 10 monclient: build_initial_monmap 2026-02-22T23:25:31.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.621+0000 7f752ca1a640 10 monclient: monmap: 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:31.623340+0000 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:31.623340+0000 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:31.626 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:31.627 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:31.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 1 Processor -- start 2026-02-22T23:25:31.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 1 -- start start 2026-02-22T23:25:31.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient: init 2026-02-22T23:25:31.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:31.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient: _add_conns ranks=[2,5,0,1,3,4] 2026-02-22T23:25:31.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f752806ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f75280c99a0 0x7f7528070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient(hunting): picked mon.noname-c con 0x7f75280c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f752815a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75280c7e90 con 0x7f752815a5e0 2026-02-22T23:25:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75280c8170 con 0x7f752806ff20 2026-02-22T23:25:31.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7528159a40 con 0x7f75280c99a0 2026-02-22T23:25:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.622+0000 7f752ca1a640 10 monclient(hunting): authenticate will time out at 1206.350701s 2026-02-22T23:25:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.623+0000 7f7526d76640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f752815a5e0 0x7f752814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35282/0 (socket says 10.20.193.6:35282) 2026-02-22T23:25:31.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.623+0000 7f7526d76640 1 -- 10.20.193.6:0/486829801 learned_addr learned my addr 10.20.193.6:0/486829801 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.623+0000 7f7525d74640 1 --2- 10.20.193.6:0/486829801 >> v2:10.20.193.80:3300/0 conn(0x7f75280c99a0 0x7f7528070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.623+0000 7f7525d74640 10 monclient(hunting): get_auth_request con 0x7f75280c99a0 auth_method 0 2026-02-22T23:25:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.623+0000 7f7525d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.623+0000 7f7525d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:31.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.623+0000 7f7525d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525573640 1 -- 10.20.193.6:0/486829801 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2164136587 0 0) 0x7f75280c7e90 con 0x7f752815a5e0 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525573640 10 monclient(hunting): _negotiate 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525573640 10 monclient(hunting): my global_id is 4785 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.624+0000 7f7525573640 1 -- 10.20.193.6:0/486829801 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7514003620 con 0x7f752815a5e0 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 10 monclient(hunting): handle_auth_done global_id 4892 payload 274 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 1 -- 10.20.193.6:0/486829801 >> v1:10.20.193.80:6789/0 conn(0x7f752806ff20 legacy=0x7f7528159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:31.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 1 -- 10.20.193.6:0/486829801 >> v1:10.20.193.6:6789/0 conn(0x7f752815a5e0 legacy=0x7f752814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:31.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 10 monclient: _finish_hunting 0 2026-02-22T23:25:31.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 1 monclient: found mon.noname-c 2026-02-22T23:25:31.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:31.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:31.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.626+0000 7f7525d74640 1 -- 10.20.193.6:0/486829801 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75280c8450 con 0x7f75280c99a0 2026-02-22T23:25:31.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.628+0000 7f7525d74640 1 --2- 10.20.193.6:0/486829801 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75280c99a0 0x7f7528070300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f75100098d0 tx=0x7f751002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f25f32c2c68d470 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 1 -- 10.20.193.6:0/486829801 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751003c070 con 0x7f75280c99a0 2026-02-22T23:25:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:31.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: dump: 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: _finish_auth 0 2026-02-22T23:25:31.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: _check_auth_tickets 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 5 monclient: authenticate success, global_id 4892 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 1 -- 10.20.193.6:0/486829801 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751002fba0 con 0x7f75280c99a0 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 1 -- 10.20.193.6:0/486829801 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510037440 con 0x7f75280c99a0 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f7525573640 10 monclient: dump: 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:31.641 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 4 set_mon_vals no callback set 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 10 monclient: shutdown 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:31.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 1 -- 10.20.193.6:0/486829801 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75280c99a0 msgr2=0x7f7528070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 1 --2- 10.20.193.6:0/486829801 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75280c99a0 0x7f7528070300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f75100098d0 tx=0x7f751002eed0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 1 -- 10.20.193.6:0/486829801 shutdown_connections 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 1 --2- 10.20.193.6:0/486829801 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75280c99a0 0x7f7528070300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 1 -- 10.20.193.6:0/486829801 >> 10.20.193.6:0/486829801 conn(0x7f75280cd780 msgr2=0x7f75280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 1 -- 10.20.193.6:0/486829801 shutdown_connections 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.630+0000 7f752ca1a640 1 -- 10.20.193.6:0/486829801 wait complete. 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient: build_initial_monmap 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient: monmap: 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:31.643 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 librados: starting msgr at 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 librados: starting objecter 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 Processor -- start 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 -- start start 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 librados: setting wanted keys 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 librados: calling monclient init 2026-02-22T23:25:31.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient: init 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f752806ff20 0x7f75280bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): picked mon.noname-c con 0x7f752806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75280c99a0 0x7f752806d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): picked mon.noname-b con 0x7f75280c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f752815a5e0 0x7f752806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): picked mon.noname-a con 0x7f752815a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7528150520 con 0x7f752815a5e0 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:31.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75280c8a10 con 0x7f75280c99a0 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f75281506a0 con 0x7f752806ff20 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.631+0000 7f752ca1a640 10 monclient(hunting): authenticate will time out at 1206.359831s 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f752815a5e0 0x7f752806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f752806ff20 0x7f75280bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f752815a5e0 0x7f752806d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54166/0 (socket says 10.20.193.6:54166) 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 1 -- 10.20.193.6:0/4111564843 learned_addr learned my addr 10.20.193.6:0/4111564843 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 10 monclient(hunting): get_auth_request con 0x7f752815a5e0 auth_method 0 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:31.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 10 monclient(hunting): get_auth_request con 0x7f752806ff20 auth_method 0 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75280c99a0 0x7f752806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7526575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:31.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 10 monclient(hunting): get_auth_request con 0x7f75280c99a0 auth_method 0 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.632+0000 7f7525d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 10 monclient(hunting): handle_auth_done global_id 4788 payload 824 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 1 -- 10.20.193.6:0/4111564843 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f752806ff20 msgr2=0x7f75280bcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f752806ff20 0x7f75280bcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 1 -- 10.20.193.6:0/4111564843 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75280c99a0 msgr2=0x7f752806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75280c99a0 0x7f752806d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 10 monclient: _finish_hunting 0 2026-02-22T23:25:31.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 1 monclient: found mon.noname-a 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.634+0000 7f7526d76640 1 -- 10.20.193.6:0/4111564843 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752806de00 con 0x7f752815a5e0 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.635+0000 7f7526d76640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f752815a5e0 0x7f752806d7c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f751800cad0 tx=0x7f751800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.635+0000 7f750effd640 10 client.?.objecter ms_handle_connect 0x7f752815a5e0 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.635+0000 7f750effd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 1 -- 10.20.193.6:0/4111564843 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751800ac50 con 0x7f752815a5e0 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: dump: 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:31.649 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: _finish_auth 0 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:31.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 5 monclient: authenticate success, global_id 4788 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 monclient: _renew_subs 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752806e0e0 con 0x7f752815a5e0 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 client.4788.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 monclient: _renew_subs 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752806e6a0 con 0x7f752815a5e0 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 1 librados: init done 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7528070300 con 0x7f752815a5e0 2026-02-22T23:25:31.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 1 -- 10.20.193.6:0/4111564843 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f751800adf0 con 0x7f752815a5e0 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 1 -- 10.20.193.6:0/4111564843 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518025630 con 0x7f752815a5e0 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f750effd640 10 monclient: dump: 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:31.652 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:31.653 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:31.653 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:31.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.636+0000 7f7524d72640 4 set_mon_vals no callback set 2026-02-22T23:25:31.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.741+0000 7f750effd640 1 -- 10.20.193.6:0/4111564843 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7518025890 con 0x7f752815a5e0 2026-02-22T23:25:31.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.741+0000 7f750effd640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74f4078e00 0x7f74f407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:31.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.742+0000 7f7526575640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74f4078e00 0x7f74f407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:31.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.742+0000 7f7526575640 10 monclient: get_auth_request con 0x7f74f4078e00 auth_method 0 2026-02-22T23:25:31.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.742+0000 7f750effd640 10 client.4788.objecter ms_handle_connect 0x7f74f4078e00 2026-02-22T23:25:31.745 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.742+0000 7f7526575640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74f4078e00 0x7f74f407b2c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f751c002410 tx=0x7f751c007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 1 -- 10.20.193.6:0/4111564843 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f7518098070 con 0x7f752815a5e0 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 10 client.4788.objecter ms_dispatch2 0x7f7528000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 3 client.4788.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 3 client.4788.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 20 client.4788.objecter dump_active .. 0 homeless 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 1 -- 10.20.193.6:0/4111564843 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7518062840 con 0x7f752815a5e0 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:31.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.750+0000 7f750effd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:31.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.854+0000 7f752ca1a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:31.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.854+0000 7f752ca1a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:31.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.854+0000 7f752ca1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:31.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.854+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f75280c8ef0 con 0x7f752815a5e0 2026-02-22T23:25:31.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.860+0000 7f750effd640 1 -- 10.20.193.6:0/4111564843 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7518066b40 con 0x7f752815a5e0 2026-02-22T23:25:31.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.860+0000 7f750effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:31.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.860+0000 7f750effd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:31.863 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:31.864 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:31.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 10 librados: watch_flush enter 2026-02-22T23:25:31.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 10 librados: watch_flush exit 2026-02-22T23:25:31.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 20 client.4788.objecter shutdown clearing up homeless session... 2026-02-22T23:25:31.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 10 client.4788.objecter successfully canceled tick 2026-02-22T23:25:31.866 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74f4078e00 msgr2=0x7f74f407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:31.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74f4078e00 0x7f74f407b2c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f751c002410 tx=0x7f751c007ef0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 10 monclient: shutdown 2026-02-22T23:25:31.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:31.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f752815a5e0 msgr2=0x7f752806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:31.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f752815a5e0 0x7f752806d7c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f751800cad0 tx=0x7f751800a060 comp rx=0 tx=0).stop 2026-02-22T23:25:31.867 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 shutdown_connections 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f74f4078e00 0x7f74f407b2c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f752815a5e0 0x7f752806d7c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75280c99a0 0x7f752806d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 --2- 10.20.193.6:0/4111564843 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f752806ff20 0x7f75280bcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 >> 10.20.193.6:0/4111564843 conn(0x7f75280cd780 msgr2=0x7f752814e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 shutdown_connections 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.861+0000 7f752ca1a640 1 -- 10.20.193.6:0/4111564843 wait complete. 2026-02-22T23:25:31.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:31.862+0000 7f752ca1a640 1 librados: shutdown 2026-02-22T23:25:34.874 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.943+0000 7fcfb3257640 10 monclient: get_monmap_and_config 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.943+0000 7fcfb3257640 10 monclient: build_initial_monmap 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.943+0000 7fcfb3257640 10 monclient: monmap: 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:34.944343+0000 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:34.944343+0000 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:34.945 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 1 Processor -- start 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 1 -- start start 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient: init 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient: _add_conns ranks=[0,5,3,4,1,2] 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcfac06ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fcfac0c99a0 0x7fcfac070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient(hunting): picked mon.noname-c con 0x7fcfac0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcfac15a5e0 0x7fcfac14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient(hunting): picked mon.noname-a con 0x7fcfac15a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfac0c7e90 con 0x7fcfac06ff20 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcfac159a40 con 0x7fcfac15a5e0 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcfac0c8170 con 0x7fcfac0c99a0 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.944+0000 7fcfb3257640 10 monclient(hunting): authenticate will time out at 1209.671719s 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfb0fcc640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fcfac06ff20 0x7fcfac159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35290/0 (socket says 10.20.193.6:35290) 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfb0fcc640 1 -- 10.20.193.6:0/1396306157 learned_addr learned my addr 10.20.193.6:0/1396306157 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:34.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 1 --2- 10.20.193.6:0/1396306157 >> v2:10.20.193.80:3300/0 conn(0x7fcfac0c99a0 0x7fcfac070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 10 monclient(hunting): get_auth_request con 0x7fcfac0c99a0 auth_method 0 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.945+0000 7fcfa3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfb17cd640 1 --2- 10.20.193.6:0/1396306157 >> v2:10.20.193.6:3300/0 conn(0x7fcfac15a5e0 0x7fcfac14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa37fe640 1 -- 10.20.193.6:0/1396306157 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352319252 0 0) 0x7fcfac0c7e90 con 0x7fcfac06ff20 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa37fe640 10 monclient(hunting): _negotiate 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa37fe640 10 monclient(hunting): my global_id is 4791 2026-02-22T23:25:34.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa37fe640 1 -- 10.20.193.6:0/1396306157 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf98003620 con 0x7fcfac06ff20 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfb17cd640 10 monclient(hunting): get_auth_request con 0x7fcfac15a5e0 auth_method 0 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfb17cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfb17cd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfb17cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 10 monclient(hunting): handle_auth_done global_id 4901 payload 274 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 1 -- 10.20.193.6:0/1396306157 >> v2:10.20.193.6:3300/0 conn(0x7fcfac15a5e0 msgr2=0x7fcfac14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 1 --2- 10.20.193.6:0/1396306157 >> v2:10.20.193.6:3300/0 conn(0x7fcfac15a5e0 0x7fcfac14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 1 -- 10.20.193.6:0/1396306157 >> v1:10.20.193.6:6789/0 conn(0x7fcfac06ff20 legacy=0x7fcfac159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 1 monclient: found mon.noname-c 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.946+0000 7fcfa3fff640 1 -- 10.20.193.6:0/1396306157 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac0c8450 con 0x7fcfac0c99a0 2026-02-22T23:25:34.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.947+0000 7fcfa3fff640 1 --2- 10.20.193.6:0/1396306157 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 0x7fcfac070300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf940098a0 tx=0x7fcf9402ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c153792599228c22 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 1 -- 10.20.193.6:0/1396306157 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9403c070 con 0x7fcfac0c99a0 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: dump: 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:34.950 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: _finish_auth 0 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: _check_auth_tickets 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 5 monclient: authenticate success, global_id 4901 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 1 -- 10.20.193.6:0/1396306157 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9402fae0 con 0x7fcfac0c99a0 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 1 -- 10.20.193.6:0/1396306157 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94037440 con 0x7fcfac0c99a0 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfa37fe640 10 monclient: dump: 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.951 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 4 set_mon_vals no callback set 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 10 monclient: shutdown 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 1 -- 10.20.193.6:0/1396306157 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 msgr2=0x7fcfac070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 1 --2- 10.20.193.6:0/1396306157 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 0x7fcfac070300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcf940098a0 tx=0x7fcf9402ee10 comp rx=0 tx=0).stop 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 1 -- 10.20.193.6:0/1396306157 shutdown_connections 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 1 --2- 10.20.193.6:0/1396306157 >> v2:10.20.193.6:3300/0 conn(0x7fcfac15a5e0 0x7fcfac14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 1 --2- 10.20.193.6:0/1396306157 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 0x7fcfac070300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 1 -- 10.20.193.6:0/1396306157 >> 10.20.193.6:0/1396306157 conn(0x7fcfac0cd780 msgr2=0x7fcfac0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.949+0000 7fcfb3257640 1 -- 10.20.193.6:0/1396306157 shutdown_connections 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 1 -- 10.20.193.6:0/1396306157 wait complete. 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 10 monclient: build_initial_monmap 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 10 monclient: monmap: 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:34.952 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 1 librados: starting msgr at 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 1 librados: starting objecter 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 1 Processor -- start 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 1 -- start start 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 1 librados: setting wanted keys 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 1 librados: calling monclient init 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.950+0000 7fcfb3257640 10 monclient: init 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:25:34.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcfac06ff20 0x7fcfac1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): picked mon.noname-a con 0x7fcfac06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 0x7fcfac1f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): picked mon.noname-c con 0x7fcfac0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcfac15a5e0 0x7fcfac1f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): picked mon.noname-b con 0x7fcfac15a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac150350 con 0x7fcfac06ff20 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac0c8a10 con 0x7fcfac15a5e0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1504d0 con 0x7fcfac0c99a0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb3257640 10 monclient(hunting): authenticate will time out at 1209.678221s 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcfac06ff20 0x7fcfac1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 0x7fcfac1f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcfac06ff20 0x7fcfac1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54190/0 (socket says 10.20.193.6:54190) 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 1 -- 10.20.193.6:0/2430270756 learned_addr learned my addr 10.20.193.6:0/2430270756 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 10 monclient(hunting): get_auth_request con 0x7fcfac06ff20 auth_method 0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 10 monclient(hunting): get_auth_request con 0x7fcfac0c99a0 auth_method 0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfb0fcc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.951+0000 7fcfa3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcfac15a5e0 0x7fcfac1f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 10 monclient(hunting): get_auth_request con 0x7fcfac15a5e0 auth_method 0 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:34.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.952+0000 7fcfb17cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 10 monclient(hunting): handle_auth_done global_id 4797 payload 824 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 1 -- 10.20.193.6:0/2430270756 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 msgr2=0x7fcfac1f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 0x7fcfac1f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 1 -- 10.20.193.6:0/2430270756 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcfac15a5e0 msgr2=0x7fcfac1f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcfac15a5e0 0x7fcfac1f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 10 monclient: _finish_hunting 0 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 1 monclient: found mon.noname-a 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.953+0000 7fcfb0fcc640 1 -- 10.20.193.6:0/2430270756 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac151ec0 con 0x7fcfac06ff20 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.954+0000 7fcfb0fcc640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcfac06ff20 0x7fcfac1534e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c760 tx=0x7fcf9c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.954+0000 7fcfa17fa640 10 client.?.objecter ms_handle_connect 0x7fcfac06ff20 2026-02-22T23:25:34.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.954+0000 7fcfa17fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 1 -- 10.20.193.6:0/2430270756 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c017070 con 0x7fcfac06ff20 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: dump: 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: _finish_auth 0 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: _check_auth_tickets 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 1 -- 10.20.193.6:0/2430270756 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c005bb0 con 0x7fcfac06ff20 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 1 -- 10.20.193.6:0/2430270756 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c007070 con 0x7fcfac06ff20 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa17fa640 10 monclient: dump: 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:34.957 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfa2ffd640 4 set_mon_vals no callback set 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 5 monclient: authenticate success, global_id 4797 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 10 monclient: _renew_subs 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac152160 con 0x7fcfac06ff20 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 10 client.4797.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 10 monclient: _renew_subs 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac152720 con 0x7fcfac06ff20 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.955+0000 7fcfb3257640 1 librados: init done 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.956+0000 7fcfb3257640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.956+0000 7fcfb3257640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.956+0000 7fcfb3257640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:34.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.956+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfac1573f0 con 0x7fcfac06ff20 2026-02-22T23:25:34.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.968+0000 7fcfa17fa640 1 -- 10.20.193.6:0/2430270756 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fcf9c007210 con 0x7fcfac06ff20 2026-02-22T23:25:34.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.968+0000 7fcfa17fa640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf7c078e80 0x7fcf7c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:34.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.969+0000 7fcfa3fff640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf7c078e80 0x7fcf7c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:34.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.969+0000 7fcfa3fff640 10 monclient: get_auth_request con 0x7fcf7c078e80 auth_method 0 2026-02-22T23:25:34.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.969+0000 7fcfa3fff640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf7c078e80 0x7fcf7c07b340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf94039000 tx=0x7fcf94033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:34.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.969+0000 7fcfa17fa640 10 client.4797.objecter ms_handle_connect 0x7fcf7c078e80 2026-02-22T23:25:34.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.970+0000 7fcfa17fa640 1 -- 10.20.193.6:0/2430270756 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fcf9c05d470 con 0x7fcfac06ff20 2026-02-22T23:25:34.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.970+0000 7fcfa17fa640 10 client.4797.objecter ms_dispatch2 0x7fcfac000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:34.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.970+0000 7fcfa17fa640 3 client.4797.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:34.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.970+0000 7fcfa17fa640 3 client.4797.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:34.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:34.970+0000 7fcfa17fa640 20 client.4797.objecter dump_active .. 0 homeless 2026-02-22T23:25:35.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.066+0000 7fcfa17fa640 1 -- 10.20.193.6:0/2430270756 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcf9c021020 con 0x7fcfac06ff20 2026-02-22T23:25:35.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.066+0000 7fcfa17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:35.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.066+0000 7fcfa17fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:35.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.168+0000 7fcfb3257640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:35.170 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.168+0000 7fcfb3257640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:35.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.168+0000 7fcfb3257640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:35.171 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.168+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcfac0c4c50 con 0x7fcfac06ff20 2026-02-22T23:25:35.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfa17fa640 1 -- 10.20.193.6:0/2430270756 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fcf9c061430 con 0x7fcfac06ff20 2026-02-22T23:25:35.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfa17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:35.173 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfa17fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:35.173 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:35.173 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 10 librados: watch_flush enter 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 10 librados: watch_flush exit 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 20 client.4797.objecter shutdown clearing up homeless session... 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 10 client.4797.objecter successfully canceled tick 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf7c078e80 msgr2=0x7fcf7c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf7c078e80 0x7fcf7c07b340 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf94039000 tx=0x7fcf94033000 comp rx=0 tx=0).stop 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 10 monclient: shutdown 2026-02-22T23:25:35.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcfac06ff20 msgr2=0x7fcfac1534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcfac06ff20 0x7fcfac1534e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c760 tx=0x7fcf9c00cc30 comp rx=0 tx=0).stop 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 shutdown_connections 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf7c078e80 0x7fcf7c07b340 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcfac15a5e0 0x7fcfac1f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcfac0c99a0 0x7fcfac1f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 --2- 10.20.193.6:0/2430270756 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcfac06ff20 0x7fcfac1534e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 >> 10.20.193.6:0/2430270756 conn(0x7fcfac0cd780 msgr2=0x7fcfac06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:35.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 shutdown_connections 2026-02-22T23:25:35.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.171+0000 7fcfb3257640 1 -- 10.20.193.6:0/2430270756 wait complete. 2026-02-22T23:25:35.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:35.172+0000 7fcfb3257640 1 librados: shutdown 2026-02-22T23:25:38.182 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:38.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.248+0000 7f1e8f6c4640 10 monclient: get_monmap_and_config 2026-02-22T23:25:38.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.248+0000 7f1e8f6c4640 10 monclient: build_initial_monmap 2026-02-22T23:25:38.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.248+0000 7f1e8f6c4640 10 monclient: monmap: 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:38.249262+0000 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:38.249262+0000 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:38.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 1 Processor -- start 2026-02-22T23:25:38.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 1 -- start start 2026-02-22T23:25:38.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient: init 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient: _add_conns ranks=[2,1,0,5,4,3] 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1e8806ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1e880c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1e8815a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e880c7e90 con 0x7f1e8815a5e0 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e880c8170 con 0x7f1e880c99a0 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e880c8450 con 0x7f1e8806ff20 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8f6c4640 10 monclient(hunting): authenticate will time out at 1212.976625s 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8dc3a640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f1e8815a5e0 0x7f1e880c6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:49980/0 (socket says 10.20.193.6:49980) 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.249+0000 7f1e8dc3a640 1 -- 10.20.193.6:0/644678174 learned_addr learned my addr 10.20.193.6:0/644678174 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2582652857 0 0) 0x7f1e880c7e90 con 0x7f1e8815a5e0 2026-02-22T23:25:38.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _negotiate 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): my global_id is 4800 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e70003620 con 0x7f1e8815a5e0 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1282624369 0 0) 0x7f1e880c8450 con 0x7f1e8806ff20 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _negotiate 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): my global_id is 4907 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e880c7e90 con 0x7f1e8806ff20 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2747057459 0 0) 0x7f1e880c8170 con 0x7f1e880c99a0 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _negotiate 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 10 monclient(hunting): my global_id is 4798 2026-02-22T23:25:38.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.251+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e880c8450 con 0x7f1e880c99a0 2026-02-22T23:25:38.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3425108903 0 0) 0x7f1e880c7e90 con 0x7f1e8806ff20 2026-02-22T23:25:38.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 >> v1:10.20.193.40:6789/0 conn(0x7f1e880c99a0 legacy=0x7f1e8814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 >> v1:10.20.193.6:6789/0 conn(0x7f1e8815a5e0 legacy=0x7f1e880c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:38.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:25:38.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880c8730 con 0x7f1e8806ff20 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: _finish_auth 0 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: _check_auth_tickets 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f1e78002ee0 con 0x7f1e8806ff20 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: dump: 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.257 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 >> v1:10.20.193.80:6789/0 conn(0x7f1e8806ff20 legacy=0x7f1e88159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70003f70 0x7f1e70004350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): picked mon.c con 0x7f1e70003f70 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:38.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e70004a10 0x7f1e70004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): picked mon.a con 0x7f1e70004a10 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e70009ab0 0x7f1e7000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): picked mon.b con 0x7f1e70009ab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7000c4c0 con 0x7f1e70004a10 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7000c670 con 0x7f1e70009ab0 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7000c870 con 0x7f1e70003f70 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.253+0000 7f1e77fff640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:25:38.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e70004a10 0x7f1e70004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8dc3a640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e70009ab0 0x7f1e7000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 10 monclient(hunting): get_auth_request con 0x7f1e70004a10 auth_method 0 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8dc3a640 10 monclient(hunting): get_auth_request con 0x7f1e70009ab0 auth_method 0 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8dc3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8dc3a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.260 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8dc3a640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8d439640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70003f70 0x7f1e70004350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.254+0000 7f1e8cc38640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8dc3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8dc3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8dc3a640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8d439640 10 monclient(hunting): get_auth_request con 0x7f1e70003f70 auth_method 0 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8d439640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8d439640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8d439640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:38.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8d439640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:38.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8d439640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:38.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8d439640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:38.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 10 monclient(hunting): handle_auth_done global_id 4907 payload 293 2026-02-22T23:25:38.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 1 -- 10.20.193.6:0/644678174 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70003f70 msgr2=0x7f1e70004350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70003f70 0x7f1e70004350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 1 -- 10.20.193.6:0/644678174 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e70009ab0 msgr2=0x7f1e7000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e70009ab0 0x7f1e7000bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 10 monclient: _finish_hunting 0 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 1 monclient: found mon.a 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8dc3a640 1 -- 10.20.193.6:0/644678174 reap_dead start 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 1 -- 10.20.193.6:0/644678174 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880c7e90 con 0x7f1e70004a10 2026-02-22T23:25:38.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 10 monclient: _finish_auth 0 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 10 monclient: _check_auth_tickets 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8cc38640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8f6c4640 5 monclient: authenticate success, global_id 4907 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.255+0000 7f1e8f6c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.256+0000 7f1e8cc38640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e70004a10 0x7f1e70004e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c003170 tx=0x7f1e7c02ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb0b1f736278450 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c002dc0 con 0x7f1e70004a10 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 10 monclient: dump: 2026-02-22T23:25:38.270 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:38.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7c0052d0 con 0x7f1e70004a10 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 1 -- 10.20.193.6:0/644678174 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0036f0 con 0x7f1e70004a10 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e77fff640 10 monclient: dump: 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.272 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 4 set_mon_vals no callback set 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 10 monclient: shutdown 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:38.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/644678174 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e70004a10 msgr2=0x7f1e70004e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e70004a10 0x7f1e70004e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c003170 tx=0x7f1e7c02ee90 comp rx=0 tx=0).stop 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/644678174 shutdown_connections 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/644678174 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e70004a10 0x7f1e70004e90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/644678174 >> 10.20.193.6:0/644678174 conn(0x7f1e880cd780 msgr2=0x7f1e880cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/644678174 shutdown_connections 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.258+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/644678174 wait complete. 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient: build_initial_monmap 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient: monmap: 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:38.274 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 librados: starting msgr at 2026-02-22T23:25:38.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 librados: starting objecter 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 Processor -- start 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 -- start start 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 librados: setting wanted keys 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 librados: calling monclient init 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient: init 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70004a10 0x7f1e881f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): picked mon.noname-c con 0x7f1e70004a10 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e88148a00 0x7f1e880bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:38.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): picked mon.noname-b con 0x7f1e88148a00 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e880bf620 0x7f1e880bda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): picked mon.noname-a con 0x7f1e880bf620 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88070300 con 0x7f1e880bf620 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88159a40 con 0x7f1e88148a00 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88070480 con 0x7f1e70004a10 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:38.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.259+0000 7f1e8f6c4640 10 monclient(hunting): authenticate will time out at 1212.986815s 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e880bf620 0x7f1e880bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e88148a00 0x7f1e880bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e880bf620 0x7f1e880bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:48490/0 (socket says 10.20.193.6:48490) 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 1 -- 10.20.193.6:0/3338277312 learned_addr learned my addr 10.20.193.6:0/3338277312 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 10 monclient(hunting): get_auth_request con 0x7f1e880bf620 auth_method 0 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70004a10 0x7f1e881f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:38.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 10 monclient(hunting): get_auth_request con 0x7f1e88148a00 auth_method 0 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8dc3a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 10 monclient(hunting): get_auth_request con 0x7f1e70004a10 auth_method 0 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8cc38640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.260+0000 7f1e8d439640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 10 monclient(hunting): handle_auth_done global_id 4803 payload 824 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 1 -- 10.20.193.6:0/3338277312 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70004a10 msgr2=0x7f1e881f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70004a10 0x7f1e881f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 1 -- 10.20.193.6:0/3338277312 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e88148a00 msgr2=0x7f1e880bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e88148a00 0x7f1e880bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 10 monclient: _finish_hunting 0 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 1 monclient: found mon.noname-a 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.261+0000 7f1e8dc3a640 1 -- 10.20.193.6:0/3338277312 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880bfaa0 con 0x7f1e880bf620 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.262+0000 7f1e8dc3a640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e880bf620 0x7f1e880bda10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e84002fd0 tx=0x7f1e8400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.262+0000 7f1e75ffb640 10 client.?.objecter ms_handle_connect 0x7f1e880bf620 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.262+0000 7f1e75ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 1 -- 10.20.193.6:0/3338277312 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e840133f0 con 0x7f1e880bf620 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:38.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: dump: 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:38.282 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: _finish_auth 0 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 5 monclient: authenticate success, global_id 4803 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 1 -- 10.20.193.6:0/3338277312 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e84019070 con 0x7f1e880bf620 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 1 -- 10.20.193.6:0/3338277312 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84013790 con 0x7f1e880bf620 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 monclient: _renew_subs 2026-02-22T23:25:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e777fe640 4 set_mon_vals no callback set 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e881f1fd0 con 0x7f1e880bf620 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e75ffb640 10 monclient: dump: 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:38.284 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 client.4803.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 monclient: _renew_subs 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e881f2540 con 0x7f1e880bf620 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 1 librados: init done 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.264+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e88159ed0 con 0x7f1e880bf620 2026-02-22T23:25:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.276+0000 7f1e75ffb640 1 -- 10.20.193.6:0/3338277312 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1e84013930 con 0x7f1e880bf620 2026-02-22T23:25:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.276+0000 7f1e75ffb640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e58078e80 0x7f1e5807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.277+0000 7f1e8d439640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e58078e80 0x7f1e5807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.277+0000 7f1e8d439640 10 monclient: get_auth_request con 0x7f1e58078e80 auth_method 0 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.277+0000 7f1e8d439640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e58078e80 0x7f1e5807b340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e780028f0 tx=0x7f1e78009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.277+0000 7f1e75ffb640 10 client.4803.objecter ms_handle_connect 0x7f1e58078e80 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.278+0000 7f1e75ffb640 1 -- 10.20.193.6:0/3338277312 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f1e8405eaa0 con 0x7f1e880bf620 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.278+0000 7f1e75ffb640 10 client.4803.objecter ms_dispatch2 0x7f1e88000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.278+0000 7f1e75ffb640 3 client.4803.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.278+0000 7f1e75ffb640 3 client.4803.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.278+0000 7f1e75ffb640 20 client.4803.objecter dump_active .. 0 homeless 2026-02-22T23:25:38.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.375+0000 7f1e75ffb640 1 -- 10.20.193.6:0/3338277312 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1e880bfaa0 con 0x7f1e880bf620 2026-02-22T23:25:38.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.375+0000 7f1e75ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:38.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.375+0000 7f1e75ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:38.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.476+0000 7f1e8f6c4640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:38.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.476+0000 7f1e8f6c4640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:38.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.476+0000 7f1e8f6c4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:38.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.476+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1e8814d5a0 con 0x7f1e880bf620 2026-02-22T23:25:38.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.478+0000 7f1e75ffb640 1 -- 10.20.193.6:0/3338277312 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1e84062a60 con 0x7f1e880bf620 2026-02-22T23:25:38.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.478+0000 7f1e75ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:38.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.478+0000 7f1e75ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:38.480 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:38.480 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 10 librados: watch_flush enter 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 10 librados: watch_flush exit 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 20 client.4803.objecter shutdown clearing up homeless session... 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 10 client.4803.objecter successfully canceled tick 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e58078e80 msgr2=0x7f1e5807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e58078e80 0x7f1e5807b340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e780028f0 tx=0x7f1e78009040 comp rx=0 tx=0).stop 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 10 monclient: shutdown 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e880bf620 msgr2=0x7f1e880bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e880bf620 0x7f1e880bda10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1e84002fd0 tx=0x7f1e8400a6a0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 shutdown_connections 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e58078e80 0x7f1e5807b340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e880bf620 0x7f1e880bda10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e88148a00 0x7f1e880bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 --2- 10.20.193.6:0/3338277312 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e70004a10 0x7f1e881f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 >> 10.20.193.6:0/3338277312 conn(0x7f1e880cd780 msgr2=0x7f1e88158810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 shutdown_connections 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.479+0000 7f1e8f6c4640 1 -- 10.20.193.6:0/3338277312 wait complete. 2026-02-22T23:25:38.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:38.480+0000 7f1e8f6c4640 1 librados: shutdown 2026-02-22T23:25:41.490 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient: get_monmap_and_config 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient: build_initial_monmap 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient: monmap: 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:41.557667+0000 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:41.557667+0000 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:41.558 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 1 Processor -- start 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 1 -- start start 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient: init 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient: _add_conns ranks=[2,5,0,4,3,1] 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc08006ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc0800c99a0 0x7fc080070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient(hunting): picked mon.noname-c con 0x7fc0800c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc08015a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.557+0000 7fc08564c640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0800c7e90 con 0x7fc08015a5e0 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc08564c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0800c8170 con 0x7fc08006ff20 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc08564c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc08564c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc080159a40 con 0x7fc0800c99a0 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc08564c640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc08564c640 10 monclient(hunting): authenticate will time out at 1216.285007s 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc07f7fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fc08015a5e0 0x7fc08014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:49988/0 (socket says 10.20.193.6:49988) 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc07f7fe640 1 -- 10.20.193.6:0/3812421625 learned_addr learned my addr 10.20.193.6:0/3812421625 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:41.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.558+0000 7fc07e7fc640 1 --2- 10.20.193.6:0/3812421625 >> v2:10.20.193.80:3300/0 conn(0x7fc0800c99a0 0x7fc080070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07e7fc640 10 monclient(hunting): get_auth_request con 0x7fc0800c99a0 auth_method 0 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:41.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2137602454 0 0) 0x7fc0800c7e90 con 0x7fc08015a5e0 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07dffb640 10 monclient(hunting): _negotiate 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07dffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07dffb640 10 monclient(hunting): my global_id is 4806 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.559+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc06c003620 con 0x7fc08015a5e0 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.560+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2971382371 0 0) 0x7fc0800c8170 con 0x7fc08006ff20 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.560+0000 7fc07dffb640 10 monclient(hunting): _negotiate 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.560+0000 7fc07dffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.560+0000 7fc07dffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.560+0000 7fc07dffb640 10 monclient(hunting): my global_id is 4916 2026-02-22T23:25:41.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.560+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0800c7e90 con 0x7fc08006ff20 2026-02-22T23:25:41.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4224377170 0 0) 0x7fc06c003620 con 0x7fc08015a5e0 2026-02-22T23:25:41.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 >> v2:10.20.193.80:3300/0 conn(0x7fc0800c99a0 msgr2=0x7fc080070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 --2- 10.20.193.6:0/3812421625 >> v2:10.20.193.80:3300/0 conn(0x7fc0800c99a0 0x7fc080070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 >> v1:10.20.193.80:6789/0 conn(0x7fc08006ff20 legacy=0x7fc080159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: _finish_hunting 0 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0800c8450 con 0x7fc08015a5e0 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: _finish_auth 0 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: _check_auth_tickets 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fc070003280 con 0x7fc08015a5e0 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: dump: 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 >> v1:10.20.193.6:6789/0 conn(0x7fc08015a5e0 legacy=0x7fc08014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc06c003e00 0x7fc06c004200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): picked mon.a con 0x7fc06c003e00 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc06c004900 0x7fc06c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): picked mon.c con 0x7fc06c004900 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc06c00bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): picked mon.b con 0x7fc06c005490 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc080159a40 con 0x7fc06c003e00 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c00c3a0 con 0x7fc06c005490 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:41.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c00c5a0 con 0x7fc06c004900 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.561+0000 7fc07dffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc06c00bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 10 monclient(hunting): get_auth_request con 0x7fc06c005490 auth_method 0 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.562+0000 7fc07f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc06c003e00 0x7fc06c004200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 10 monclient(hunting): get_auth_request con 0x7fc06c003e00 auth_method 0 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:41.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.563+0000 7fc07effd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 10 monclient(hunting): handle_auth_done global_id 4806 payload 293 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 1 -- 10.20.193.6:0/3812421625 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc06c004900 msgr2=0x7fc06c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc06c004900 0x7fc06c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 1 -- 10.20.193.6:0/3812421625 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc06c003e00 msgr2=0x7fc06c004200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc06c003e00 0x7fc06c004200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 1 monclient: found mon.b 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 1 -- 10.20.193.6:0/3812421625 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c003620 con 0x7fc06c005490 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 10 monclient: _finish_auth 0 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc08564c640 5 monclient: authenticate success, global_id 4806 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc08564c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:41.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.564+0000 7fc07f7fe640 1 -- 10.20.193.6:0/3812421625 reap_dead start 2026-02-22T23:25:41.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.565+0000 7fc07f7fe640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc06c00bac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc07000a770 tx=0x7fc07002fb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a872ed8980d5510 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07000a390 con 0x7fc06c005490 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 10 monclient: dump: 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc070033400 con 0x7fc06c005490 2026-02-22T23:25:41.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 1 -- 10.20.193.6:0/3812421625 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc070033810 con 0x7fc06c005490 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 4 set_mon_vals no callback set 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc07dffb640 10 monclient: dump: 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 10 monclient: shutdown 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 1 -- 10.20.193.6:0/3812421625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 msgr2=0x7fc06c00bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc06c00bac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc07000a770 tx=0x7fc07002fb10 comp rx=0 tx=0).stop 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 1 -- 10.20.193.6:0/3812421625 shutdown_connections 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 1 --2- 10.20.193.6:0/3812421625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc06c00bac0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 1 -- 10.20.193.6:0/3812421625 >> 10.20.193.6:0/3812421625 conn(0x7fc0800cd780 msgr2=0x7fc0800cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 1 -- 10.20.193.6:0/3812421625 shutdown_connections 2026-02-22T23:25:41.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 1 -- 10.20.193.6:0/3812421625 wait complete. 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 10 monclient: build_initial_monmap 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.567+0000 7fc08564c640 10 monclient: monmap: 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 librados: starting msgr at 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 librados: starting objecter 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 Processor -- start 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 -- start start 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 librados: setting wanted keys 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 librados: calling monclient init 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient: init 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc0801f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): picked mon.noname-b con 0x7fc06c005490 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:41.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc0801f7b30 0x7fc080148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): picked mon.noname-c con 0x7fc0801f7b30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc080148cd0 0x7fc0801584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): picked mon.noname-a con 0x7fc080148cd0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc080070300 con 0x7fc080148cd0 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800c8a10 con 0x7fc06c005490 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc080070480 con 0x7fc0801f7b30 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc08564c640 10 monclient(hunting): authenticate will time out at 1216.295650s 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.568+0000 7fc07f7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc080148cd0 0x7fc0801584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc0801f7b30 0x7fc080148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc0801f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc080148cd0 0x7fc0801584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:48518/0 (socket says 10.20.193.6:48518) 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 1 -- 10.20.193.6:0/3200316076 learned_addr learned my addr 10.20.193.6:0/3200316076 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 10 monclient(hunting): get_auth_request con 0x7fc080148cd0 auth_method 0 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 10 monclient(hunting): get_auth_request con 0x7fc06c005490 auth_method 0 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 10 monclient(hunting): get_auth_request con 0x7fc0801f7b30 auth_method 0 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:41.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.569+0000 7fc07e7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 10 monclient(hunting): handle_auth_done global_id 4919 payload 824 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 1 -- 10.20.193.6:0/3200316076 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 msgr2=0x7fc0801f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc0801f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 1 -- 10.20.193.6:0/3200316076 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc080148cd0 msgr2=0x7fc0801584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc080148cd0 0x7fc0801584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 1 monclient: found mon.noname-c 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07e7fc640 1 -- 10.20.193.6:0/3200316076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08006dc60 con 0x7fc0801f7b30 2026-02-22T23:25:41.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.570+0000 7fc07effd640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc0801f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:25:41.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.571+0000 7fc07e7fc640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc0801f7b30 0x7fc080148790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e820 tx=0x7fc06800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:41.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.571+0000 7fc05f7fe640 10 client.?.objecter ms_handle_connect 0x7fc0801f7b30 2026-02-22T23:25:41.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.571+0000 7fc05f7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 1 -- 10.20.193.6:0/3200316076 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068017070 con 0x7fc0801f7b30 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: dump: 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: _finish_auth 0 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 5 monclient: authenticate success, global_id 4919 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 10 monclient: _renew_subs 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08006df40 con 0x7fc0801f7b30 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 10 client.4919.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 10 monclient: _renew_subs 2026-02-22T23:25:41.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08006e500 con 0x7fc0801f7b30 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 1 librados: init done 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 1 -- 10.20.193.6:0/3200316076 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06800ee90 con 0x7fc0801f7b30 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 1 -- 10.20.193.6:0/3200316076 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068020060 con 0x7fc0801f7b30 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc05f7fe640 10 monclient: dump: 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc07d7fa640 4 set_mon_vals no callback set 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.573+0000 7fc08564c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.574+0000 7fc08564c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.574+0000 7fc08564c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.574+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc080159ed0 con 0x7fc0801f7b30 2026-02-22T23:25:41.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.584+0000 7fc05f7fe640 1 -- 10.20.193.6:0/3200316076 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc06800cb40 con 0x7fc0801f7b30 2026-02-22T23:25:41.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.585+0000 7fc05f7fe640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc04c078e00 0x7fc04c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:41.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.585+0000 7fc07effd640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc04c078e00 0x7fc04c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:41.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.585+0000 7fc07effd640 10 monclient: get_auth_request con 0x7fc04c078e00 auth_method 0 2026-02-22T23:25:41.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.586+0000 7fc07effd640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc04c078e00 0x7fc04c07b2c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc074005720 tx=0x7fc074007d60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:41.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.586+0000 7fc05f7fe640 10 client.4919.objecter ms_handle_connect 0x7fc04c078e00 2026-02-22T23:25:41.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.586+0000 7fc05f7fe640 1 -- 10.20.193.6:0/3200316076 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc0680977f0 con 0x7fc0801f7b30 2026-02-22T23:25:41.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.586+0000 7fc05f7fe640 10 client.4919.objecter ms_dispatch2 0x7fc080000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:41.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.586+0000 7fc05f7fe640 3 client.4919.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:41.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.586+0000 7fc05f7fe640 3 client.4919.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:41.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.586+0000 7fc05f7fe640 20 client.4919.objecter dump_active .. 0 homeless 2026-02-22T23:25:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.684+0000 7fc05f7fe640 1 -- 10.20.193.6:0/3200316076 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc06800ce60 con 0x7fc0801f7b30 2026-02-22T23:25:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.684+0000 7fc05f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.684+0000 7fc05f7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.787+0000 7fc08564c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.787+0000 7fc08564c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.787+0000 7fc08564c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.787+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc08014d5a0 con 0x7fc0801f7b30 2026-02-22T23:25:41.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.790+0000 7fc05f7fe640 1 -- 10.20.193.6:0/3200316076 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc068061eb0 con 0x7fc0801f7b30 2026-02-22T23:25:41.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.790+0000 7fc05f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:41.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.790+0000 7fc05f7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:41.792 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:41.792 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:41.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 10 librados: watch_flush enter 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 10 librados: watch_flush exit 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 20 client.4919.objecter shutdown clearing up homeless session... 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 10 client.4919.objecter successfully canceled tick 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc04c078e00 msgr2=0x7fc04c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc04c078e00 0x7fc04c07b2c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc074005720 tx=0x7fc074007d60 comp rx=0 tx=0).stop 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 10 monclient: shutdown 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc0801f7b30 msgr2=0x7fc080148790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc0801f7b30 0x7fc080148790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e820 tx=0x7fc06800ecf0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 shutdown_connections 2026-02-22T23:25:41.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc04c078e00 0x7fc04c07b2c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc080148cd0 0x7fc0801584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc0801f7b30 0x7fc080148790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 --2- 10.20.193.6:0/3200316076 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc06c005490 0x7fc0801f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:41.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 >> 10.20.193.6:0/3200316076 conn(0x7fc0800cd780 msgr2=0x7fc0800cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:41.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 shutdown_connections 2026-02-22T23:25:41.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 -- 10.20.193.6:0/3200316076 wait complete. 2026-02-22T23:25:41.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:41.791+0000 7fc08564c640 1 librados: shutdown 2026-02-22T23:25:44.801 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:44.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.868+0000 7f9f7db2c640 10 monclient: get_monmap_and_config 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.868+0000 7f9f7db2c640 10 monclient: build_initial_monmap 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.868+0000 7f9f7db2c640 10 monclient: monmap: 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:44.869015+0000 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:44.869015+0000 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:44.870 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.868+0000 7f9f7db2c640 1 Processor -- start 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.868+0000 7f9f7db2c640 1 -- start start 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.868+0000 7f9f7db2c640 10 monclient: init 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient: _add_conns ranks=[2,1,4,3,5,0] 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f9f7806ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f9f780c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f9f7815a5e0 0x7f9f78070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient(hunting): picked mon.noname-b con 0x7f9f7815a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:25:44.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f780c7e90 con 0x7f9f780c99a0 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f780c8170 con 0x7f9f7806ff20 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9f78159a40 con 0x7f9f7815a5e0 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.869+0000 7f9f7db2c640 10 monclient(hunting): authenticate will time out at 1219.596496s 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f777fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f9f7806ff20 0x7f9f78159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:41678/0 (socket says 10.20.193.6:41678) 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f777fe640 1 -- 10.20.193.6:0/3163311248 learned_addr learned my addr 10.20.193.6:0/3163311248 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 1 --2- 10.20.193.6:0/3163311248 >> v2:10.20.193.40:3300/0 conn(0x7f9f7815a5e0 0x7f9f78070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 10 monclient(hunting): get_auth_request con 0x7f9f7815a5e0 auth_method 0 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:44.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.870+0000 7f9f77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.871+0000 7f9f767fc640 1 -- 10.20.193.6:0/3163311248 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419588610 0 0) 0x7f9f780c8170 con 0x7f9f7806ff20 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.871+0000 7f9f767fc640 10 monclient(hunting): _negotiate 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.871+0000 7f9f767fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.871+0000 7f9f767fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.871+0000 7f9f767fc640 10 monclient(hunting): my global_id is 4922 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.871+0000 7f9f767fc640 1 -- 10.20.193.6:0/3163311248 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f5c003620 con 0x7f9f7806ff20 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 10 monclient(hunting): handle_auth_done global_id 4807 payload 274 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 1 -- 10.20.193.6:0/3163311248 >> v1:10.20.193.80:6789/0 conn(0x7f9f7806ff20 legacy=0x7f9f78159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 1 -- 10.20.193.6:0/3163311248 >> v1:10.20.193.40:6789/0 conn(0x7f9f780c99a0 legacy=0x7f9f7814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:44.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:44.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 1 monclient: found mon.noname-b 2026-02-22T23:25:44.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:44.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:25:44.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.872+0000 7f9f77fff640 1 -- 10.20.193.6:0/3163311248 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f780c8450 con 0x7f9f7815a5e0 2026-02-22T23:25:44.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.873+0000 7f9f77fff640 1 --2- 10.20.193.6:0/3163311248 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f7815a5e0 0x7f9f78070300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009dc0 tx=0x7f9f6c02f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=812f382b8166cf21 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f767fc640 1 -- 10.20.193.6:0/3163311248 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c03c070 con 0x7f9f7815a5e0 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f767fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f767fc640 10 monclient: dump: 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:44.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f767fc640 10 monclient: _finish_auth 0 2026-02-22T23:25:44.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f767fc640 10 monclient: _check_auth_tickets 2026-02-22T23:25:44.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f767fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:44.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f7db2c640 5 monclient: authenticate success, global_id 4807 2026-02-22T23:25:44.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.874+0000 7f9f7db2c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f767fc640 1 -- 10.20.193.6:0/3163311248 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c037440 con 0x7f9f7815a5e0 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f767fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f767fc640 1 -- 10.20.193.6:0/3163311248 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0377f0 con 0x7f9f7815a5e0 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f767fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f767fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f767fc640 10 monclient: dump: 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f7db2c640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.875+0000 7f9f7db2c640 4 set_mon_vals no callback set 2026-02-22T23:25:44.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 10 monclient: shutdown 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 -- 10.20.193.6:0/3163311248 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f7815a5e0 msgr2=0x7f9f78070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/3163311248 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f7815a5e0 0x7f9f78070300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009dc0 tx=0x7f9f6c02f280 comp rx=0 tx=0).stop 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 -- 10.20.193.6:0/3163311248 shutdown_connections 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/3163311248 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f7815a5e0 0x7f9f78070300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 -- 10.20.193.6:0/3163311248 >> 10.20.193.6:0/3163311248 conn(0x7f9f780cd780 msgr2=0x7f9f780cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 -- 10.20.193.6:0/3163311248 shutdown_connections 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 -- 10.20.193.6:0/3163311248 wait complete. 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 10 monclient: build_initial_monmap 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 10 monclient: monmap: 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 librados: starting msgr at 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 librados: starting objecter 2026-02-22T23:25:44.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 Processor -- start 2026-02-22T23:25:44.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 -- start start 2026-02-22T23:25:44.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 librados: setting wanted keys 2026-02-22T23:25:44.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 1 librados: calling monclient init 2026-02-22T23:25:44.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.876+0000 7f9f7db2c640 10 monclient: init 2026-02-22T23:25:44.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:44.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9f7806ff20 0x7f9f7814c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): picked mon.noname-a con 0x7f9f7806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f780c99a0 0x7f9f780c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): picked mon.noname-b con 0x7f9f780c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9f7815a5e0 0x7f9f781f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): picked mon.noname-c con 0x7f9f7815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78150520 con 0x7f9f7806ff20 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9f780c8a10 con 0x7f9f780c99a0 2026-02-22T23:25:44.885 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9f781506a0 con 0x7f9f7815a5e0 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f7db2c640 10 monclient(hunting): authenticate will time out at 1219.604298s 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f77fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9f7815a5e0 0x7f9f781f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f76ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f780c99a0 0x7f9f780c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f777fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9f7806ff20 0x7f9f7814c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f76ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f780c99a0 0x7f9f780c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:35036/0 (socket says 10.20.193.6:35036) 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f76ffd640 1 -- 10.20.193.6:0/1161213906 learned_addr learned my addr 10.20.193.6:0/1161213906 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f76ffd640 10 monclient(hunting): get_auth_request con 0x7f9f780c99a0 auth_method 0 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:44.886 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f77fff640 10 monclient(hunting): get_auth_request con 0x7f9f7815a5e0 auth_method 0 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f77fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f77fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f77fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f777fe640 10 monclient(hunting): get_auth_request con 0x7f9f7806ff20 auth_method 0 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f777fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.877+0000 7f9f777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f76ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f76ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f76ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f77fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:44.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f77fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f77fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.878+0000 7f9f777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 10 monclient(hunting): handle_auth_done global_id 4810 payload 824 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 1 -- 10.20.193.6:0/1161213906 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9f7815a5e0 msgr2=0x7f9f781f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9f7815a5e0 0x7f9f781f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 1 -- 10.20.193.6:0/1161213906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9f7806ff20 msgr2=0x7f9f7814c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9f7806ff20 0x7f9f7814c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 1 monclient: found mon.noname-b 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:44.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.879+0000 7f9f76ffd640 1 -- 10.20.193.6:0/1161213906 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f780c1270 con 0x7f9f780c99a0 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.880+0000 7f9f76ffd640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f780c99a0 0x7f9f780c25f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c760 tx=0x7f9f6800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.880+0000 7f9f53fff640 10 client.?.objecter ms_handle_connect 0x7f9f780c99a0 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.880+0000 7f9f53fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 1 -- 10.20.193.6:0/1161213906 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680052d0 con 0x7f9f780c99a0 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: dump: 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.889 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: _finish_auth 0 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: _check_auth_tickets 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 1 -- 10.20.193.6:0/1161213906 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f68005470 con 0x7f9f780c99a0 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 1 -- 10.20.193.6:0/1161213906 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680038d0 con 0x7f9f780c99a0 2026-02-22T23:25:44.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f75ffb640 4 set_mon_vals no callback set 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f53fff640 10 monclient: dump: 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:44.891 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 5 monclient: authenticate success, global_id 4810 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 monclient: _renew_subs 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f780c1550 con 0x7f9f780c99a0 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 client.4810.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 monclient: _renew_subs 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f780c1b10 con 0x7f9f780c99a0 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 1 librados: init done 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:44.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.882+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f78070300 con 0x7f9f780c99a0 2026-02-22T23:25:44.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.894+0000 7f9f53fff640 1 -- 10.20.193.6:0/1161213906 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9f68003a70 con 0x7f9f780c99a0 2026-02-22T23:25:44.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.894+0000 7f9f53fff640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9f48078e00 0x7f9f4807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:44.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.895+0000 7f9f777fe640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9f48078e00 0x7f9f4807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:44.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.895+0000 7f9f777fe640 10 monclient: get_auth_request con 0x7f9f48078e00 auth_method 0 2026-02-22T23:25:44.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.895+0000 7f9f777fe640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9f48078e00 0x7f9f4807b2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005980 tx=0x7f9f64007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:44.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.895+0000 7f9f53fff640 10 client.4810.objecter ms_handle_connect 0x7f9f48078e00 2026-02-22T23:25:44.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.896+0000 7f9f53fff640 1 -- 10.20.193.6:0/1161213906 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f9f6805e0a0 con 0x7f9f780c99a0 2026-02-22T23:25:44.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.896+0000 7f9f53fff640 10 client.4810.objecter ms_dispatch2 0x7f9f78000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:44.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.896+0000 7f9f53fff640 3 client.4810.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:44.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.896+0000 7f9f53fff640 3 client.4810.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:44.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.896+0000 7f9f53fff640 20 client.4810.objecter dump_active .. 0 homeless 2026-02-22T23:25:45.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.999+0000 7f9f53fff640 1 -- 10.20.193.6:0/1161213906 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f9f68062060 con 0x7f9f780c99a0 2026-02-22T23:25:45.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.999+0000 7f9f53fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:45.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:44.999+0000 7f9f53fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:45.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.102+0000 7f9f7db2c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:45.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.102+0000 7f9f7db2c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:45.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.102+0000 7f9f7db2c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:25:45.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.102+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9f780c4970 con 0x7f9f780c99a0 2026-02-22T23:25:45.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.105+0000 7f9f53fff640 1 -- 10.20.193.6:0/1161213906 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f9f68066360 con 0x7f9f780c99a0 2026-02-22T23:25:45.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.105+0000 7f9f53fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:45.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.105+0000 7f9f53fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:45.107 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:45.107 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:45.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 10 librados: watch_flush enter 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 10 librados: watch_flush exit 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 20 client.4810.objecter shutdown clearing up homeless session... 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 10 client.4810.objecter successfully canceled tick 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9f48078e00 msgr2=0x7f9f4807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9f48078e00 0x7f9f4807b2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9f64005980 tx=0x7f9f64007dd0 comp rx=0 tx=0).stop 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 10 monclient: shutdown 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f780c99a0 msgr2=0x7f9f780c25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f780c99a0 0x7f9f780c25f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c760 tx=0x7f9f6800cc30 comp rx=0 tx=0).stop 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 shutdown_connections 2026-02-22T23:25:45.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9f48078e00 0x7f9f4807b2c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:45.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9f7815a5e0 0x7f9f781f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:45.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9f780c99a0 0x7f9f780c25f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:45.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 --2- 10.20.193.6:0/1161213906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9f7806ff20 0x7f9f7814c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:45.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 >> 10.20.193.6:0/1161213906 conn(0x7f9f780cd780 msgr2=0x7f9f780c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:45.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 shutdown_connections 2026-02-22T23:25:45.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 -- 10.20.193.6:0/1161213906 wait complete. 2026-02-22T23:25:45.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:45.106+0000 7f9f7db2c640 1 librados: shutdown 2026-02-22T23:25:48.118 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:48.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.185+0000 7f65c8e4c640 10 monclient: get_monmap_and_config 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.185+0000 7f65c8e4c640 10 monclient: build_initial_monmap 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.185+0000 7f65c8e4c640 10 monclient: monmap: 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:48.186518+0000 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:48.186518+0000 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:48.190 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 1 Processor -- start 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 1 -- start start 2026-02-22T23:25:48.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient: init 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient: _add_conns ranks=[5,2,3,0,4,1] 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f65c406ff20 0x7f65c4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient(hunting): picked mon.noname-c con 0x7f65c406ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f65c40c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f65c415a5e0 0x7f65c414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient(hunting): picked mon.noname-a con 0x7f65c415a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c40c7e90 con 0x7f65c40c99a0 2026-02-22T23:25:48.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f65c4159a40 con 0x7f65c415a5e0 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f65c40c8170 con 0x7f65c406ff20 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.186+0000 7f65c8e4c640 10 monclient(hunting): authenticate will time out at 1222.913900s 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f65c415a5e0 0x7f65c414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2575640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f65c406ff20 0x7f65c4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c1d74640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f65c40c99a0 0x7f65c4159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:59438/0 (socket says 10.20.193.6:59438) 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c1d74640 1 -- 10.20.193.6:0/1396092187 learned_addr learned my addr 10.20.193.6:0/1396092187 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 10 monclient(hunting): get_auth_request con 0x7f65c415a5e0 auth_method 0 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:48.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2575640 10 monclient(hunting): get_auth_request con 0x7f65c406ff20 auth_method 0 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.187+0000 7f65c2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:48.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 10 monclient(hunting): handle_auth_done global_id 4815 payload 274 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 1 -- 10.20.193.6:0/1396092187 >> v2:10.20.193.80:3300/0 conn(0x7f65c406ff20 msgr2=0x7f65c4070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 1 --2- 10.20.193.6:0/1396092187 >> v2:10.20.193.80:3300/0 conn(0x7f65c406ff20 0x7f65c4070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 1 -- 10.20.193.6:0/1396092187 >> v1:10.20.193.80:6789/0 conn(0x7f65c40c99a0 legacy=0x7f65c4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 10 monclient: _finish_hunting 0 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 1 monclient: found mon.noname-a 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.188+0000 7f65c2d76640 1 -- 10.20.193.6:0/1396092187 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c40c8450 con 0x7f65c415a5e0 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.189+0000 7f65c2d76640 1 --2- 10.20.193.6:0/1396092187 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c415a5e0 0x7f65c414fe10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009d90 tx=0x7f65b402f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0a9ab3863c4c88 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 1 -- 10.20.193.6:0/1396092187 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b403c070 con 0x7f65c415a5e0 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: dump: 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:48.201 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: _finish_auth 0 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: _check_auth_tickets 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 1 -- 10.20.193.6:0/1396092187 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b4037440 con 0x7f65c415a5e0 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c8e4c640 5 monclient: authenticate success, global_id 4815 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c8e4c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 1 -- 10.20.193.6:0/1396092187 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4037810 con 0x7f65c415a5e0 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c1573640 10 monclient: dump: 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:48.202 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c8e4c640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.191+0000 7f65c8e4c640 4 set_mon_vals no callback set 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 10 monclient: shutdown 2026-02-22T23:25:48.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 -- 10.20.193.6:0/1396092187 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c415a5e0 msgr2=0x7f65c414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/1396092187 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c415a5e0 0x7f65c414fe10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009d90 tx=0x7f65b402f1e0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 -- 10.20.193.6:0/1396092187 shutdown_connections 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/1396092187 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c415a5e0 0x7f65c414fe10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/1396092187 >> v2:10.20.193.80:3300/0 conn(0x7f65c406ff20 0x7f65c4070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 -- 10.20.193.6:0/1396092187 >> 10.20.193.6:0/1396092187 conn(0x7f65c40cd780 msgr2=0x7f65c40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 -- 10.20.193.6:0/1396092187 shutdown_connections 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 -- 10.20.193.6:0/1396092187 wait complete. 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 10 monclient: build_initial_monmap 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 10 monclient: monmap: 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:48.204 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 librados: starting msgr at 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 librados: starting objecter 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 Processor -- start 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 -- start start 2026-02-22T23:25:48.205 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 librados: setting wanted keys 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 1 librados: calling monclient init 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.192+0000 7f65c8e4c640 10 monclient: init 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c406ff20 0x7f65c41f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): picked mon.noname-a con 0x7f65c406ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65c40c99a0 0x7f65c40c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): picked mon.noname-c con 0x7f65c40c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65c415a5e0 0x7f65c41f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): picked mon.noname-b con 0x7f65c415a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:48.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4150350 con 0x7f65c406ff20 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f65c40c8a10 con 0x7f65c415a5e0 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41504d0 con 0x7f65c40c99a0 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c8e4c640 10 monclient(hunting): authenticate will time out at 1222.920314s 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c406ff20 0x7f65c41f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c406ff20 0x7f65c41f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:45180/0 (socket says 10.20.193.6:45180) 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 1 -- 10.20.193.6:0/3983989986 learned_addr learned my addr 10.20.193.6:0/3983989986 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 10 monclient(hunting): get_auth_request con 0x7f65c406ff20 auth_method 0 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:48.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c1d74640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65c40c99a0 0x7f65c40c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c2d76640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65c415a5e0 0x7f65c41f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c1d74640 10 monclient(hunting): get_auth_request con 0x7f65c40c99a0 auth_method 0 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c1d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c1d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.193+0000 7f65c1d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:48.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c2d76640 10 monclient(hunting): get_auth_request con 0x7f65c415a5e0 auth_method 0 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c2d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c1d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c1d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c1d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.194+0000 7f65c2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 10 monclient(hunting): handle_auth_done global_id 4818 payload 824 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 1 -- 10.20.193.6:0/3983989986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65c40c99a0 msgr2=0x7f65c40c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:48.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65c40c99a0 0x7f65c40c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 1 -- 10.20.193.6:0/3983989986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65c415a5e0 msgr2=0x7f65c41f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65c415a5e0 0x7f65c41f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 10 monclient: _finish_hunting 0 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 1 monclient: found mon.noname-a 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.195+0000 7f65c2575640 1 -- 10.20.193.6:0/3983989986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c41f21a0 con 0x7f65c406ff20 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.196+0000 7f65c2575640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c406ff20 0x7f65c41f5690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f65b800ebe0 tx=0x7f65b800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.196+0000 7f65aaffd640 10 client.?.objecter ms_handle_connect 0x7f65c406ff20 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.196+0000 7f65aaffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 1 -- 10.20.193.6:0/3983989986 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8017070 con 0x7f65c406ff20 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:48.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: dump: 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: _finish_auth 0 2026-02-22T23:25:48.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 1 -- 10.20.193.6:0/3983989986 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b800cc40 con 0x7f65c406ff20 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 1 -- 10.20.193.6:0/3983989986 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8013470 con 0x7f65c406ff20 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65aaffd640 10 monclient: dump: 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:48.212 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 5 monclient: authenticate success, global_id 4818 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c0d72640 4 set_mon_vals no callback set 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 monclient: _renew_subs 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c41f2480 con 0x7f65c406ff20 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 client.4818.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 monclient: _renew_subs 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c40ba7c0 con 0x7f65c406ff20 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 1 librados: init done 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:48.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.198+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c40ba300 con 0x7f65c406ff20 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.210+0000 7f65aaffd640 1 -- 10.20.193.6:0/3983989986 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f65b8013610 con 0x7f65c406ff20 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.210+0000 7f65aaffd640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6590078e80 0x7f659007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.210+0000 7f65c1d74640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6590078e80 0x7f659007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.211+0000 7f65c1d74640 10 monclient: get_auth_request con 0x7f6590078e80 auth_method 0 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.211+0000 7f65aaffd640 10 client.4818.objecter ms_handle_connect 0x7f6590078e80 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.211+0000 7f65c1d74640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6590078e80 0x7f659007b340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65c414d5e0 tx=0x7f65ac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.212+0000 7f65aaffd640 1 -- 10.20.193.6:0/3983989986 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f65b805e510 con 0x7f65c406ff20 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.212+0000 7f65aaffd640 10 client.4818.objecter ms_dispatch2 0x7f65c4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.212+0000 7f65aaffd640 3 client.4818.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.212+0000 7f65aaffd640 3 client.4818.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:48.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.212+0000 7f65aaffd640 20 client.4818.objecter dump_active .. 0 homeless 2026-02-22T23:25:48.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.311+0000 7f65aaffd640 1 -- 10.20.193.6:0/3983989986 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f65c40ba300 con 0x7f65c406ff20 2026-02-22T23:25:48.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.311+0000 7f65aaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:48.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.311+0000 7f65aaffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:48.417 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.416+0000 7f65c8e4c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:48.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.416+0000 7f65c8e4c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:48.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.416+0000 7f65c8e4c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:48.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.416+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f65c4150860 con 0x7f65c406ff20 2026-02-22T23:25:48.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.418+0000 7f65aaffd640 1 -- 10.20.193.6:0/3983989986 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f65b80624d0 con 0x7f65c406ff20 2026-02-22T23:25:48.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.418+0000 7f65aaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:48.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.418+0000 7f65aaffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:48.420 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:48.420 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 10 librados: watch_flush enter 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 10 librados: watch_flush exit 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 20 client.4818.objecter shutdown clearing up homeless session... 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 10 client.4818.objecter successfully canceled tick 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6590078e80 msgr2=0x7f659007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6590078e80 0x7f659007b340 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65c414d5e0 tx=0x7f65ac009040 comp rx=0 tx=0).stop 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 10 monclient: shutdown 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c406ff20 msgr2=0x7f65c41f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.419+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c406ff20 0x7f65c41f5690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f65b800ebe0 tx=0x7f65b800c520 comp rx=0 tx=0).stop 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 shutdown_connections 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6590078e80 0x7f659007b340 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f65c415a5e0 0x7f65c41f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f65c40c99a0 0x7f65c40c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 --2- 10.20.193.6:0/3983989986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f65c406ff20 0x7f65c41f5690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 >> 10.20.193.6:0/3983989986 conn(0x7f65c40cd780 msgr2=0x7f65c414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 shutdown_connections 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 -- 10.20.193.6:0/3983989986 wait complete. 2026-02-22T23:25:48.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:48.420+0000 7f65c8e4c640 1 librados: shutdown 2026-02-22T23:25:51.431 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:51.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.498+0000 7f1e2496a640 10 monclient: get_monmap_and_config 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.498+0000 7f1e2496a640 10 monclient: build_initial_monmap 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.498+0000 7f1e2496a640 10 monclient: monmap: 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:51.499494+0000 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:51.499494+0000 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:51.501 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 1 Processor -- start 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 1 -- start start 2026-02-22T23:25:51.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient: init 2026-02-22T23:25:51.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:51.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient: _add_conns ranks=[2,3,0,4,5,1] 2026-02-22T23:25:51.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1e1c06ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:51.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1e1c0c99a0 0x7f1e1c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:51.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient(hunting): picked mon.noname-a con 0x7f1e1c0c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:51.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1e1c15a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:51.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1c0c7e90 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1c0c8170 con 0x7f1e1c06ff20 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1c159a40 con 0x7f1e1c0c99a0 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.499+0000 7f1e2496a640 10 monclient(hunting): authenticate will time out at 1226.226855s 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.500+0000 7f1e21ede640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1e1c0c99a0 0x7f1e1c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.500+0000 7f1e22ee0640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f1e1c15a5e0 0x7f1e1c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:33926/0 (socket says 10.20.193.6:33926) 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.500+0000 7f1e22ee0640 1 -- 10.20.193.6:0/1562193143 learned_addr learned my addr 10.20.193.6:0/1562193143 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.500+0000 7f1e21ede640 10 monclient(hunting): get_auth_request con 0x7f1e1c0c99a0 auth_method 0 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.500+0000 7f1e21ede640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.500+0000 7f1e21ede640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:51.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.500+0000 7f1e21ede640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.501+0000 7f1e21ede640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.501+0000 7f1e21ede640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.501+0000 7f1e21ede640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e216dd640 1 -- 10.20.193.6:0/1562193143 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185782693 0 0) 0x7f1e1c0c8170 con 0x7f1e1c06ff20 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e216dd640 10 monclient(hunting): _negotiate 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e216dd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e216dd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e216dd640 10 monclient(hunting): my global_id is 4937 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e216dd640 1 -- 10.20.193.6:0/1562193143 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e0c003620 con 0x7f1e1c06ff20 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e21ede640 10 monclient(hunting): handle_auth_done global_id 4821 payload 274 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e21ede640 1 -- 10.20.193.6:0/1562193143 >> v1:10.20.193.80:6789/0 conn(0x7f1e1c06ff20 legacy=0x7f1e1c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e21ede640 1 -- 10.20.193.6:0/1562193143 >> v1:10.20.193.6:6789/0 conn(0x7f1e1c15a5e0 legacy=0x7f1e1c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:51.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e21ede640 10 monclient: _finish_hunting 0 2026-02-22T23:25:51.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.502+0000 7f1e21ede640 1 monclient: found mon.noname-a 2026-02-22T23:25:51.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.503+0000 7f1e21ede640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:51.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.503+0000 7f1e21ede640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:51.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.503+0000 7f1e21ede640 1 -- 10.20.193.6:0/1562193143 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c0c8450 con 0x7f1e1c0c99a0 2026-02-22T23:25:51.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.504+0000 7f1e21ede640 1 --2- 10.20.193.6:0/1562193143 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c0c99a0 0x7f1e1c070300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098d0 tx=0x7f1e0802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3df6a53d2018492 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:51.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 1 -- 10.20.193.6:0/1562193143 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0803c070 con 0x7f1e1c0c99a0 2026-02-22T23:25:51.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:51.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:51.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: dump: 2026-02-22T23:25:51.513 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:51.513 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: _finish_auth 0 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:51.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 1 -- 10.20.193.6:0/1562193143 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0802fba0 con 0x7f1e1c0c99a0 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 5 monclient: authenticate success, global_id 4821 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 1 -- 10.20.193.6:0/1562193143 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08037440 con 0x7f1e1c0c99a0 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 4 set_mon_vals no callback set 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e216dd640 10 monclient: dump: 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.515 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 10 monclient: shutdown 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 1 -- 10.20.193.6:0/1562193143 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c0c99a0 msgr2=0x7f1e1c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 1 --2- 10.20.193.6:0/1562193143 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c0c99a0 0x7f1e1c070300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098d0 tx=0x7f1e0802eed0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 1 -- 10.20.193.6:0/1562193143 shutdown_connections 2026-02-22T23:25:51.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 1 --2- 10.20.193.6:0/1562193143 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c0c99a0 0x7f1e1c070300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 1 -- 10.20.193.6:0/1562193143 >> 10.20.193.6:0/1562193143 conn(0x7f1e1c0cd780 msgr2=0x7f1e1c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 1 -- 10.20.193.6:0/1562193143 shutdown_connections 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.506+0000 7f1e2496a640 1 -- 10.20.193.6:0/1562193143 wait complete. 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 10 monclient: build_initial_monmap 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 10 monclient: monmap: 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:51.517 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 1 librados: starting msgr at 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 1 librados: starting objecter 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 1 Processor -- start 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 1 -- start start 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 1 librados: setting wanted keys 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 1 librados: calling monclient init 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.507+0000 7f1e2496a640 10 monclient: init 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e1c06ff20 0x7f1e1c14bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:51.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): picked mon.noname-c con 0x7f1e1c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e1c0c99a0 0x7f1e1c156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): picked mon.noname-b con 0x7f1e1c0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c15a5e0 0x7f1e1c156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): picked mon.noname-a con 0x7f1e1c15a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c150520 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c0c8a10 con 0x7f1e1c0c99a0 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1506a0 con 0x7f1e1c06ff20 2026-02-22T23:25:51.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e2496a640 10 monclient(hunting): authenticate will time out at 1226.235122s 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c15a5e0 0x7f1e1c156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c15a5e0 0x7f1e1c156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:45188/0 (socket says 10.20.193.6:45188) 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 1 -- 10.20.193.6:0/2844379523 learned_addr learned my addr 10.20.193.6:0/2844379523 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 10 monclient(hunting): get_auth_request con 0x7f1e1c15a5e0 auth_method 0 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:51.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e226df640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e1c06ff20 0x7f1e1c14bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e21ede640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e1c0c99a0 0x7f1e1c156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e22ee0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e226df640 10 monclient(hunting): get_auth_request con 0x7f1e1c06ff20 auth_method 0 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e226df640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e226df640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e226df640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e21ede640 10 monclient(hunting): get_auth_request con 0x7f1e1c0c99a0 auth_method 0 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e21ede640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e21ede640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.508+0000 7f1e21ede640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:51.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.509+0000 7f1e226df640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.509+0000 7f1e226df640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.509+0000 7f1e226df640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.509+0000 7f1e21ede640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.509+0000 7f1e21ede640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.509+0000 7f1e21ede640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 10 monclient(hunting): handle_auth_done global_id 4827 payload 824 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 1 -- 10.20.193.6:0/2844379523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e1c06ff20 msgr2=0x7f1e1c14bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e1c06ff20 0x7f1e1c14bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 1 -- 10.20.193.6:0/2844379523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e1c0c99a0 msgr2=0x7f1e1c156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e1c0c99a0 0x7f1e1c156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 10 monclient: _finish_hunting 0 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 1 monclient: found mon.noname-a 2026-02-22T23:25:51.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:51.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:51.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.510+0000 7f1e22ee0640 1 -- 10.20.193.6:0/2844379523 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c06ef20 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.511+0000 7f1e22ee0640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c15a5e0 0x7f1e1c156990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000cb10 tx=0x7f1e1000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:51.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.511+0000 7f1e06ffd640 10 client.?.objecter ms_handle_connect 0x7f1e1c15a5e0 2026-02-22T23:25:51.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.511+0000 7f1e06ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:51.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 1 -- 10.20.193.6:0/2844379523 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10017070 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: dump: 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:51.525 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: _finish_auth 0 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 1 -- 10.20.193.6:0/2844379523 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1000ac50 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 5 monclient: authenticate success, global_id 4827 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 10 monclient: _renew_subs 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c06f200 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 1 -- 10.20.193.6:0/2844379523 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10012400 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e20edc640 4 set_mon_vals no callback set 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e06ffd640 10 monclient: dump: 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:51.527 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 10 client.4827.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 10 monclient: _renew_subs 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c155690 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.512+0000 7f1e2496a640 1 librados: init done 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.513+0000 7f1e2496a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.513+0000 7f1e2496a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.513+0000 7f1e2496a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:51.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.513+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1c1573f0 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.524+0000 7f1e06ffd640 1 -- 10.20.193.6:0/2844379523 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1e10024080 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.525+0000 7f1e06ffd640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1dec078e00 0x7f1dec07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.525+0000 7f1e226df640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1dec078e00 0x7f1dec07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.525+0000 7f1e226df640 10 monclient: get_auth_request con 0x7f1dec078e00 auth_method 0 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.525+0000 7f1e06ffd640 10 client.4827.objecter ms_handle_connect 0x7f1dec078e00 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.525+0000 7f1e226df640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1dec078e00 0x7f1dec07b2c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e14002960 tx=0x7f1e140028f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.526+0000 7f1e06ffd640 1 -- 10.20.193.6:0/2844379523 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f1e1005dd80 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.526+0000 7f1e06ffd640 10 client.4827.objecter ms_dispatch2 0x7f1e1c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.526+0000 7f1e06ffd640 3 client.4827.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.526+0000 7f1e06ffd640 3 client.4827.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:51.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.526+0000 7f1e06ffd640 20 client.4827.objecter dump_active .. 0 homeless 2026-02-22T23:25:51.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.624+0000 7f1e06ffd640 1 -- 10.20.193.6:0/2844379523 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1e10061d40 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.624+0000 7f1e06ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:51.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.624+0000 7f1e06ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:51.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.723+0000 7f1e2496a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:51.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.723+0000 7f1e2496a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:51.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.723+0000 7f1e2496a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:51.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.723+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1e1c0c4a10 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.725+0000 7f1e06ffd640 1 -- 10.20.193.6:0/2844379523 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1e10066040 con 0x7f1e1c15a5e0 2026-02-22T23:25:51.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.725+0000 7f1e06ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:51.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.725+0000 7f1e06ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:51.727 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:51.727 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 10 librados: watch_flush enter 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 10 librados: watch_flush exit 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 20 client.4827.objecter shutdown clearing up homeless session... 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 10 client.4827.objecter successfully canceled tick 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1dec078e00 msgr2=0x7f1dec07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1dec078e00 0x7f1dec07b2c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e14002960 tx=0x7f1e140028f0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 10 monclient: shutdown 2026-02-22T23:25:51.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c15a5e0 msgr2=0x7f1e1c156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c15a5e0 0x7f1e1c156990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000cb10 tx=0x7f1e1000a060 comp rx=0 tx=0).stop 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 shutdown_connections 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1dec078e00 0x7f1dec07b2c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e1c15a5e0 0x7f1e1c156990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e1c0c99a0 0x7f1e1c156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 --2- 10.20.193.6:0/2844379523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e1c06ff20 0x7f1e1c14bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 >> 10.20.193.6:0/2844379523 conn(0x7f1e1c0cd780 msgr2=0x7f1e1c14e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 shutdown_connections 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 -- 10.20.193.6:0/2844379523 wait complete. 2026-02-22T23:25:51.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:51.726+0000 7f1e2496a640 1 librados: shutdown 2026-02-22T23:25:53.061 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:04:37.311 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.090 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:04:37.336 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.091 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:04:37.338 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.091 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:04:37.341 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.102 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:04:37.349 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.103 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:04:37.353 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.117 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:04:37.328 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.118 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:04:37.330 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:53.129 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:04:37.340 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:25:54.737 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:54.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.806+0000 7f0f83111640 10 monclient: get_monmap_and_config 2026-02-22T23:25:54.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.806+0000 7f0f83111640 10 monclient: build_initial_monmap 2026-02-22T23:25:54.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.806+0000 7f0f83111640 10 monclient: monmap: 2026-02-22T23:25:54.808 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:54.807058+0000 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:54.807058+0000 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:54.809 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:54.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.806+0000 7f0f83111640 1 Processor -- start 2026-02-22T23:25:54.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.806+0000 7f0f83111640 1 -- start start 2026-02-22T23:25:54.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.806+0000 7f0f83111640 10 monclient: init 2026-02-22T23:25:54.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:54.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient: _add_conns ranks=[0,5,3,1,4,2] 2026-02-22T23:25:54.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0f7c1527d0 addr v1:10.20.193.6:6789/0 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f0f7c152bb0 0x7f0f7c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient(hunting): picked mon.noname-c con 0x7f0f7c152bb0 addr v2:10.20.193.80:3300/0 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f0f7c0c5fc0 0x7f0f7c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient(hunting): picked mon.noname-a con 0x7f0f7c0c5fc0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f7c06e300 con 0x7f0f7c1527d0 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7c06f1f0 con 0x7f0f7c0c5fc0 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7c0c6470 con 0x7f0f7c152bb0 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f83111640 10 monclient(hunting): authenticate will time out at 1229.534518s 2026-02-22T23:25:54.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.807+0000 7f0f81687640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f0f7c0c5fc0 0x7f0f7c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f80e86640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f0f7c1527d0 0x7f0f7c06da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:33942/0 (socket says 10.20.193.6:33942) 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f80e86640 1 -- 10.20.193.6:0/4166812751 learned_addr learned my addr 10.20.193.6:0/4166812751 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 1 --2- 10.20.193.6:0/4166812751 >> v2:10.20.193.80:3300/0 conn(0x7f0f7c152bb0 0x7f0f7c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f81687640 10 monclient(hunting): get_auth_request con 0x7f0f7c0c5fc0 auth_method 0 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f81687640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f81687640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f81687640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 10 monclient(hunting): get_auth_request con 0x7f0f7c152bb0 auth_method 0 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:54.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f81687640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f81687640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.808+0000 7f0f81687640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 10 monclient(hunting): handle_auth_done global_id 4943 payload 274 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 1 -- 10.20.193.6:0/4166812751 >> v2:10.20.193.6:3300/0 conn(0x7f0f7c0c5fc0 msgr2=0x7f0f7c0c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 1 --2- 10.20.193.6:0/4166812751 >> v2:10.20.193.6:3300/0 conn(0x7f0f7c0c5fc0 0x7f0f7c0c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 1 -- 10.20.193.6:0/4166812751 >> v1:10.20.193.6:6789/0 conn(0x7f0f7c1527d0 legacy=0x7f0f7c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 10 monclient: _finish_hunting 0 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 1 monclient: found mon.noname-c 2026-02-22T23:25:54.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:54.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:54.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.809+0000 7f0f73fff640 1 -- 10.20.193.6:0/4166812751 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c06e5e0 con 0x7f0f7c152bb0 2026-02-22T23:25:54.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.810+0000 7f0f73fff640 1 --2- 10.20.193.6:0/4166812751 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c0c5a80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0750e0 tx=0x7f0f6402ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab599fe964bf72f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:54.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 1 -- 10.20.193.6:0/4166812751 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6403c070 con 0x7f0f7c152bb0 2026-02-22T23:25:54.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:54.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:54.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: dump: 2026-02-22T23:25:54.822 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: _finish_auth 0 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: _check_auth_tickets 2026-02-22T23:25:54.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 1 -- 10.20.193.6:0/4166812751 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6402faf0 con 0x7f0f7c152bb0 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 1 -- 10.20.193.6:0/4166812751 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f64037440 con 0x7f0f7c152bb0 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f737fe640 10 monclient: dump: 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:54.824 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f83111640 5 monclient: authenticate success, global_id 4943 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f83111640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.812+0000 7f0f83111640 4 set_mon_vals no callback set 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 10 monclient: shutdown 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 -- 10.20.193.6:0/4166812751 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 msgr2=0x7f0f7c0c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 --2- 10.20.193.6:0/4166812751 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c0c5a80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0750e0 tx=0x7f0f6402ede0 comp rx=0 tx=0).stop 2026-02-22T23:25:54.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 -- 10.20.193.6:0/4166812751 shutdown_connections 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 --2- 10.20.193.6:0/4166812751 >> v2:10.20.193.6:3300/0 conn(0x7f0f7c0c5fc0 0x7f0f7c0c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 --2- 10.20.193.6:0/4166812751 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c0c5a80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 -- 10.20.193.6:0/4166812751 >> 10.20.193.6:0/4166812751 conn(0x7f0f7c0cd780 msgr2=0x7f0f7c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 -- 10.20.193.6:0/4166812751 shutdown_connections 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 -- 10.20.193.6:0/4166812751 wait complete. 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 10 monclient: build_initial_monmap 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 10 monclient: monmap: 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:54.826 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 librados: starting msgr at 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.813+0000 7f0f83111640 1 librados: starting objecter 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 Processor -- start 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 -- start start 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 librados: setting wanted keys 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 librados: calling monclient init 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient: init 2026-02-22T23:25:54.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0f7c0c5fc0 0x7f0f7c0c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): picked mon.noname-a con 0x7f0f7c0c5fc0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0f7c1527d0 0x7f0f7c1f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): picked mon.noname-b con 0x7f0f7c1527d0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c1f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): picked mon.noname-c con 0x7f0f7c152bb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c06ee80 con 0x7f0f7c0c5fc0 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c06eba0 con 0x7f0f7c1527d0 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:54.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7c0c93a0 con 0x7f0f7c152bb0 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f83111640 10 monclient(hunting): authenticate will time out at 1229.541509s 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f81687640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c1f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.814+0000 7f0f80e86640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0f7c0c5fc0 0x7f0f7c0c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c1f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:46592/0 (socket says 10.20.193.6:46592) 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 1 -- 10.20.193.6:0/479547002 learned_addr learned my addr 10.20.193.6:0/479547002 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 10 monclient(hunting): get_auth_request con 0x7f0f7c152bb0 auth_method 0 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f80e86640 10 monclient(hunting): get_auth_request con 0x7f0f7c0c5fc0 auth_method 0 2026-02-22T23:25:54.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f80e86640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f80e86640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f80e86640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0f7c1527d0 0x7f0f7c1f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f81687640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f80e86640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f80e86640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f80e86640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 10 monclient(hunting): get_auth_request con 0x7f0f7c1527d0 auth_method 0 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:54.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.815+0000 7f0f73fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 10 monclient(hunting): handle_auth_done global_id 4946 payload 824 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 1 -- 10.20.193.6:0/479547002 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0f7c1527d0 msgr2=0x7f0f7c1f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0f7c1527d0 0x7f0f7c1f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 1 -- 10.20.193.6:0/479547002 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0f7c0c5fc0 msgr2=0x7f0f7c0c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0f7c0c5fc0 0x7f0f7c0c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 10 monclient: _finish_hunting 0 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 1 monclient: found mon.noname-c 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:54.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.816+0000 7f0f81687640 1 -- 10.20.193.6:0/479547002 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7c1f4610 con 0x7f0f7c152bb0 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.817+0000 7f0f81687640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c1f1cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009dd0 tx=0x7f0f7800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.817+0000 7f0f717fa640 10 client.?.objecter ms_handle_connect 0x7f0f7c152bb0 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.817+0000 7f0f717fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f717fa640 1 -- 10.20.193.6:0/479547002 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78018020 con 0x7f0f7c152bb0 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f717fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f717fa640 10 monclient: dump: 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.832 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f717fa640 10 monclient: _finish_auth 0 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f717fa640 10 monclient: _check_auth_tickets 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f717fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 5 monclient: authenticate success, global_id 4946 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 10 monclient: _renew_subs 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:54.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7c0c4240 con 0x7f0f7c152bb0 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 10 client.4946.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 10 monclient: _renew_subs 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7c1f5bf0 con 0x7f0f7c152bb0 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.818+0000 7f0f83111640 1 librados: init done 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f717fa640 1 -- 10.20.193.6:0/479547002 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f78014480 con 0x7f0f7c152bb0 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f717fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f717fa640 1 -- 10.20.193.6:0/479547002 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78014830 con 0x7f0f7c152bb0 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f717fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f717fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f717fa640 10 monclient: dump: 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:54.834 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f72ffd640 4 set_mon_vals no callback set 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f83111640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f83111640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f83111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:54.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.819+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7c0c9730 con 0x7f0f7c152bb0 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.830+0000 7f0f717fa640 1 -- 10.20.193.6:0/479547002 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0f780149d0 con 0x7f0f7c152bb0 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.830+0000 7f0f717fa640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0f4c078e00 0x7f0f4c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.830+0000 7f0f80e86640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0f4c078e00 0x7f0f4c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.830+0000 7f0f80e86640 10 monclient: get_auth_request con 0x7f0f4c078e00 auth_method 0 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.831+0000 7f0f80e86640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0f4c078e00 0x7f0f4c07b2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c002770 tx=0x7f0f6c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.831+0000 7f0f717fa640 10 client.4946.objecter ms_handle_connect 0x7f0f4c078e00 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.832+0000 7f0f717fa640 1 -- 10.20.193.6:0/479547002 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f0f7805f250 con 0x7f0f7c152bb0 2026-02-22T23:25:54.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.832+0000 7f0f717fa640 10 client.4946.objecter ms_dispatch2 0x7f0f7c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:54.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.832+0000 7f0f717fa640 3 client.4946.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:54.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.832+0000 7f0f717fa640 3 client.4946.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:54.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.832+0000 7f0f717fa640 20 client.4946.objecter dump_active .. 0 homeless 2026-02-22T23:25:54.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.930+0000 7f0f717fa640 1 -- 10.20.193.6:0/479547002 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0f78022070 con 0x7f0f7c152bb0 2026-02-22T23:25:54.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.930+0000 7f0f717fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:54.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:54.930+0000 7f0f717fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:55.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.030+0000 7f0f83111640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:55.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.030+0000 7f0f83111640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:55.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.030+0000 7f0f83111640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:25:55.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.030+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0f7c1512f0 con 0x7f0f7c152bb0 2026-02-22T23:25:55.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.032+0000 7f0f717fa640 1 -- 10.20.193.6:0/479547002 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0f78010040 con 0x7f0f7c152bb0 2026-02-22T23:25:55.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.032+0000 7f0f717fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:55.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.032+0000 7f0f717fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:55.034 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:55.034 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.033+0000 7f0f83111640 10 librados: watch_flush enter 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.033+0000 7f0f83111640 10 librados: watch_flush exit 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.033+0000 7f0f83111640 20 client.4946.objecter shutdown clearing up homeless session... 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.033+0000 7f0f83111640 10 client.4946.objecter successfully canceled tick 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0f4c078e00 msgr2=0x7f0f4c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0f4c078e00 0x7f0f4c07b2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c002770 tx=0x7f0f6c009040 comp rx=0 tx=0).stop 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 10 monclient: shutdown 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 msgr2=0x7f0f7c1f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c1f1cd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009dd0 tx=0x7f0f7800c490 comp rx=0 tx=0).stop 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 shutdown_connections 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0f4c078e00 0x7f0f4c07b2c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0f7c152bb0 0x7f0f7c1f1cd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0f7c1527d0 0x7f0f7c1f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 --2- 10.20.193.6:0/479547002 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0f7c0c5fc0 0x7f0f7c0c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 >> 10.20.193.6:0/479547002 conn(0x7f0f7c0cd780 msgr2=0x7f0f7c0c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 shutdown_connections 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 -- 10.20.193.6:0/479547002 wait complete. 2026-02-22T23:25:55.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:55.034+0000 7f0f83111640 1 librados: shutdown 2026-02-22T23:25:58.044 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:25:58.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.112+0000 7f401d016640 10 monclient: get_monmap_and_config 2026-02-22T23:25:58.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.112+0000 7f401d016640 10 monclient: build_initial_monmap 2026-02-22T23:25:58.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.112+0000 7f401d016640 10 monclient: monmap: 2026-02-22T23:25:58.114 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:58.114 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:58.114 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:25:58.112790+0000 2026-02-22T23:25:58.114 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:25:58.112790+0000 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.112+0000 7f401d016640 1 Processor -- start 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.112+0000 7f401d016640 1 -- start start 2026-02-22T23:25:58.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.112+0000 7f401d016640 10 monclient: init 2026-02-22T23:25:58.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:58.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient: _add_conns ranks=[1,3,2,4,0,5] 2026-02-22T23:25:58.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f401806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:25:58.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f4018155eb0 0x7f4018070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:58.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient(hunting): picked mon.noname-a con 0x7f4018155eb0 addr v2:10.20.193.6:3300/0 2026-02-22T23:25:58.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f401815c970 addr v1:10.20.193.80:6789/0 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40180725b0 con 0x7f401806ff20 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4018072890 con 0x7f401815c970 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f4018072b70 con 0x7f4018155eb0 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f401d016640 10 monclient(hunting): authenticate will time out at 1232.840208s 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f4016d76640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f401806ff20 0x7f40181557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:58474/0 (socket says 10.20.193.6:58474) 2026-02-22T23:25:58.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.113+0000 7f4016d76640 1 -- 10.20.193.6:0/3190727119 learned_addr learned my addr 10.20.193.6:0/3190727119 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:25:58.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.119+0000 7f4016575640 1 --2- 10.20.193.6:0/3190727119 >> v2:10.20.193.6:3300/0 conn(0x7f4018155eb0 0x7f4018070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:58.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.120+0000 7f4016575640 10 monclient(hunting): get_auth_request con 0x7f4018155eb0 auth_method 0 2026-02-22T23:25:58.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.120+0000 7f4016575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:58.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.120+0000 7f4016575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:58.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.120+0000 7f4016575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:58.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.120+0000 7f4016575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:58.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.120+0000 7f4016575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:58.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.120+0000 7f4016575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:58.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 10 monclient(hunting): handle_auth_done global_id 4839 payload 274 2026-02-22T23:25:58.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 1 -- 10.20.193.6:0/3190727119 >> v1:10.20.193.80:6789/0 conn(0x7f401815c970 legacy=0x7f40180c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:58.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 1 -- 10.20.193.6:0/3190727119 >> v1:10.20.193.40:6789/0 conn(0x7f401806ff20 legacy=0x7f40181557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:58.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 10 monclient: _finish_hunting 0 2026-02-22T23:25:58.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 1 monclient: found mon.noname-a 2026-02-22T23:25:58.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:58.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:58.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.121+0000 7f4016575640 1 -- 10.20.193.6:0/3190727119 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4018072e50 con 0x7f4018155eb0 2026-02-22T23:25:58.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.122+0000 7f4016575640 1 --2- 10.20.193.6:0/3190727119 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f4018070300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40000098a0 tx=0x7f400002ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a0ec6dc3868e9 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 1 -- 10.20.193.6:0/3190727119 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400003c070 con 0x7f4018155eb0 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: dump: 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:58.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: _finish_auth 0 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: _check_auth_tickets 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 1 -- 10.20.193.6:0/3190727119 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400002fb70 con 0x7f4018155eb0 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 5 monclient: authenticate success, global_id 4839 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 1 -- 10.20.193.6:0/3190727119 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000037440 con 0x7f4018155eb0 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 10 monclient: get_monmap_and_config success 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 4 set_mon_vals no callback set 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f4015d74640 10 monclient: dump: 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.127 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 10 monclient: shutdown 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 1 -- 10.20.193.6:0/3190727119 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 msgr2=0x7f4018070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 1 --2- 10.20.193.6:0/3190727119 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f4018070300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f40000098a0 tx=0x7f400002ee60 comp rx=0 tx=0).stop 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 1 -- 10.20.193.6:0/3190727119 shutdown_connections 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 1 --2- 10.20.193.6:0/3190727119 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f4018070300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 1 -- 10.20.193.6:0/3190727119 >> 10.20.193.6:0/3190727119 conn(0x7f40180cd780 msgr2=0x7f40180cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 1 -- 10.20.193.6:0/3190727119 shutdown_connections 2026-02-22T23:25:58.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.124+0000 7f401d016640 1 -- 10.20.193.6:0/3190727119 wait complete. 2026-02-22T23:25:58.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient: build_initial_monmap 2026-02-22T23:25:58.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient: monmap: 2026-02-22T23:25:58.130 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:25:58.130 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:25:58.130 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:25:58.130 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:25:58.130 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 librados: starting msgr at 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 librados: starting objecter 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 Processor -- start 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 -- start start 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 librados: setting wanted keys 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 librados: calling monclient init 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient: init 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient: _reopen_session rank -1 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f401806ff20 0x7f40180c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): picked mon.noname-b con 0x7f401806ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f40181f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): picked mon.noname-a con 0x7f4018155eb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:25:58.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f401815c970 0x7f40180c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): picked mon.noname-c con 0x7f401815c970 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f401815d340 con 0x7f4018155eb0 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f401815d1c0 con 0x7f401806ff20 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f401815d4c0 con 0x7f401815c970 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): _renew_subs 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.125+0000 7f401d016640 10 monclient(hunting): authenticate will time out at 1232.852863s 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f40181f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f40181f7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:51820/0 (socket says 10.20.193.6:51820) 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 1 -- 10.20.193.6:0/3171332091 learned_addr learned my addr 10.20.193.6:0/3171332091 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 10 monclient(hunting): get_auth_request con 0x7f4018155eb0 auth_method 0 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.126+0000 7f4016575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:25:58.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 10 monclient(hunting): handle_auth_done global_id 4842 payload 824 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 1 -- 10.20.193.6:0/3171332091 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f401815c970 msgr2=0x7f40180c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f401815c970 0x7f40180c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 1 -- 10.20.193.6:0/3171332091 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f401806ff20 msgr2=0x7f40180c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f401806ff20 0x7f40180c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 10 monclient: _finish_hunting 0 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 1 monclient: found mon.noname-a 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.127+0000 7f4016575640 1 -- 10.20.193.6:0/3171332091 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40180c6200 con 0x7f4018155eb0 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.128+0000 7f4016575640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f40181f7820 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4000009130 tx=0x7f40000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.128+0000 7f3ff77fe640 10 client.?.objecter ms_handle_connect 0x7f4018155eb0 2026-02-22T23:25:58.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.128+0000 7f3ff77fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 1 -- 10.20.193.6:0/3171332091 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400003c040 con 0x7f4018155eb0 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: dump: 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:58.134 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: _finish_auth 0 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 1 -- 10.20.193.6:0/3171332091 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4000041470 con 0x7f4018155eb0 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 5 monclient: authenticate success, global_id 4842 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 monclient: _renew_subs 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40181f4310 con 0x7f4018155eb0 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 1 -- 10.20.193.6:0/3171332091 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000041860 con 0x7f4018155eb0 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f3ff77fe640 10 monclient: dump: 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:25:58.135 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f4015573640 4 set_mon_vals no callback set 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 client.4842.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 monclient: _renew_subs 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40180c6470 con 0x7f4018155eb0 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 1 librados: init done 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:58.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.130+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4018160410 con 0x7f4018155eb0 2026-02-22T23:25:58.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.141+0000 7f3ff77fe640 1 -- 10.20.193.6:0/3171332091 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4000041a00 con 0x7f4018155eb0 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.141+0000 7f3ff77fe640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3fe4078e80 0x7f3fe407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f4016d76640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3fe4078e80 0x7f3fe407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f4016d76640 10 monclient: get_auth_request con 0x7f3fe4078e80 auth_method 0 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f4016d76640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3fe4078e80 0x7f3fe407b340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f400c007fd0 tx=0x7f400c007b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f3ff77fe640 10 client.4842.objecter ms_handle_connect 0x7f3fe4078e80 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f3ff77fe640 1 -- 10.20.193.6:0/3171332091 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f40000836f0 con 0x7f4018155eb0 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f3ff77fe640 10 client.4842.objecter ms_dispatch2 0x7f4018000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f3ff77fe640 3 client.4842.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:25:58.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f3ff77fe640 3 client.4842.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:25:58.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.142+0000 7f3ff77fe640 20 client.4842.objecter dump_active .. 0 homeless 2026-02-22T23:25:58.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.240+0000 7f3ff77fe640 1 -- 10.20.193.6:0/3171332091 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4018160410 con 0x7f4018155eb0 2026-02-22T23:25:58.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.240+0000 7f3ff77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:25:58.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.240+0000 7f3ff77fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:25:58.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.340+0000 7f401d016640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:25:58.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.340+0000 7f401d016640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:58.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.340+0000 7f401d016640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:25:58.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.340+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4018070300 con 0x7f4018155eb0 2026-02-22T23:25:58.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.342+0000 7f3ff77fe640 1 -- 10.20.193.6:0/3171332091 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4000087db0 con 0x7f4018155eb0 2026-02-22T23:25:58.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.342+0000 7f3ff77fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:25:58.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.342+0000 7f3ff77fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:25:58.344 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:25:58.344 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:25:58.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 10 librados: watch_flush enter 2026-02-22T23:25:58.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 10 librados: watch_flush exit 2026-02-22T23:25:58.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 20 client.4842.objecter shutdown clearing up homeless session... 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 10 client.4842.objecter successfully canceled tick 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3fe4078e80 msgr2=0x7f3fe407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3fe4078e80 0x7f3fe407b340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f400c007fd0 tx=0x7f400c007b20 comp rx=0 tx=0).stop 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 10 monclient: shutdown 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 msgr2=0x7f40181f7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f40181f7820 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4000009130 tx=0x7f40000047c0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 shutdown_connections 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3fe4078e80 0x7f3fe407b340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f401815c970 0x7f40180c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4018155eb0 0x7f40181f7820 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 --2- 10.20.193.6:0/3171332091 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f401806ff20 0x7f40180c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:25:58.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 >> 10.20.193.6:0/3171332091 conn(0x7f40180cd780 msgr2=0x7f4018150f10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:25:58.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 shutdown_connections 2026-02-22T23:25:58.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 -- 10.20.193.6:0/3171332091 wait complete. 2026-02-22T23:25:58.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:25:58.344+0000 7f401d016640 1 librados: shutdown 2026-02-22T23:26:01.355 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:01.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.422+0000 7fa12806f640 10 monclient: get_monmap_and_config 2026-02-22T23:26:01.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.422+0000 7fa12806f640 10 monclient: build_initial_monmap 2026-02-22T23:26:01.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.422+0000 7fa12806f640 10 monclient: monmap: 2026-02-22T23:26:01.424 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:01.424 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:01.424 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:01.423401+0000 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:01.423401+0000 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:01.425 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:01.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 1 Processor -- start 2026-02-22T23:26:01.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 1 -- start start 2026-02-22T23:26:01.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient: init 2026-02-22T23:26:01.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient: _add_conns ranks=[4,5,2,3,0,1] 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa12006ff20 0x7fa120070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient(hunting): picked mon.noname-b con 0x7fa12006ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa1200c99a0 0x7fa120159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient(hunting): picked mon.noname-c con 0x7fa1200c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa12015a410 addr v1:10.20.193.80:6789/0 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa12015a860 con 0x7fa12015a410 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa120159a40 con 0x7fa12006ff20 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa1200c7e90 con 0x7fa1200c99a0 2026-02-22T23:26:01.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.423+0000 7fa12806f640 10 monclient(hunting): authenticate will time out at 1236.150873s 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa125de4640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa12006ff20 0x7fa120070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa1255e3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa1200c99a0 0x7fa120159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa125de4640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa12006ff20 0x7fa120070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:49156/0 (socket says 10.20.193.6:49156) 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa125de4640 1 -- 10.20.193.6:0/286436420 learned_addr learned my addr 10.20.193.6:0/286436420 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa125de4640 10 monclient(hunting): get_auth_request con 0x7fa12006ff20 auth_method 0 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa125de4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa125de4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa125de4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa1255e3640 10 monclient(hunting): get_auth_request con 0x7fa1200c99a0 auth_method 0 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa1255e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:01.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa1255e3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.424+0000 7fa1255e3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.425+0000 7fa125de4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.425+0000 7fa125de4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.425+0000 7fa125de4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.425+0000 7fa1255e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.425+0000 7fa1255e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.425+0000 7fa1255e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 10 monclient(hunting): handle_auth_done global_id 4822 payload 274 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 1 -- 10.20.193.6:0/286436420 >> v2:10.20.193.80:3300/0 conn(0x7fa1200c99a0 msgr2=0x7fa120159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 1 --2- 10.20.193.6:0/286436420 >> v2:10.20.193.80:3300/0 conn(0x7fa1200c99a0 0x7fa120159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 1 -- 10.20.193.6:0/286436420 >> v1:10.20.193.80:6789/0 conn(0x7fa12015a410 legacy=0x7fa12014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 10 monclient: _finish_hunting 0 2026-02-22T23:26:01.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 1 monclient: found mon.noname-b 2026-02-22T23:26:01.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:01.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:01.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.426+0000 7fa125de4640 1 -- 10.20.193.6:0/286436420 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1200c8170 con 0x7fa12006ff20 2026-02-22T23:26:01.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.427+0000 7fa125de4640 1 --2- 10.20.193.6:0/286436420 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12006ff20 0x7fa120070300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa110009970 tx=0x7fa11002eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd20d0ff7672fe23 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:01.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 1 -- 10.20.193.6:0/286436420 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11003c070 con 0x7fa12006ff20 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: dump: 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:01.438 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: _finish_auth 0 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: _check_auth_tickets 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 1 -- 10.20.193.6:0/286436420 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11002fbc0 con 0x7fa12006ff20 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 5 monclient: authenticate success, global_id 4822 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 1 -- 10.20.193.6:0/286436420 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110037440 con 0x7fa12006ff20 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:01.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 4 set_mon_vals no callback set 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa124de2640 10 monclient: dump: 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:01.440 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 10 monclient: shutdown 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 -- 10.20.193.6:0/286436420 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12006ff20 msgr2=0x7fa120070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 --2- 10.20.193.6:0/286436420 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12006ff20 0x7fa120070300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa110009970 tx=0x7fa11002eef0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 -- 10.20.193.6:0/286436420 shutdown_connections 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 --2- 10.20.193.6:0/286436420 >> v2:10.20.193.80:3300/0 conn(0x7fa1200c99a0 0x7fa120159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 --2- 10.20.193.6:0/286436420 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12006ff20 0x7fa120070300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 -- 10.20.193.6:0/286436420 >> 10.20.193.6:0/286436420 conn(0x7fa1200cd780 msgr2=0x7fa1200cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 -- 10.20.193.6:0/286436420 shutdown_connections 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.429+0000 7fa12806f640 1 -- 10.20.193.6:0/286436420 wait complete. 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 10 monclient: build_initial_monmap 2026-02-22T23:26:01.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 10 monclient: monmap: 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 1 librados: starting msgr at 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 1 librados: starting objecter 2026-02-22T23:26:01.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 1 Processor -- start 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 1 -- start start 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 1 librados: setting wanted keys 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 1 librados: calling monclient init 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.430+0000 7fa12806f640 10 monclient: init 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa12006ff20 0x7fa12014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): picked mon.noname-a con 0x7fa12006ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1200c99a0 0x7fa1201f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): picked mon.noname-c con 0x7fa1200c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12015a410 0x7fa1201f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:01.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): picked mon.noname-b con 0x7fa12015a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200c8a10 con 0x7fa12006ff20 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200c8730 con 0x7fa12015a410 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa120150520 con 0x7fa1200c99a0 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa12806f640 10 monclient(hunting): authenticate will time out at 1236.158081s 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa12006ff20 0x7fa12014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12015a410 0x7fa1201f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1255e3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1200c99a0 0x7fa1201f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:01.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12015a410 0x7fa1201f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:49172/0 (socket says 10.20.193.6:49172) 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 1 -- 10.20.193.6:0/3579611582 learned_addr learned my addr 10.20.193.6:0/3579611582 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 10 monclient(hunting): get_auth_request con 0x7fa12015a410 auth_method 0 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 10 monclient(hunting): get_auth_request con 0x7fa12006ff20 auth_method 0 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1255e3640 10 monclient(hunting): get_auth_request con 0x7fa1200c99a0 auth_method 0 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1255e3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:01.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1255e3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1255e3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa1265e5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.431+0000 7fa125de4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.432+0000 7fa1255e3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.432+0000 7fa1255e3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.432+0000 7fa1255e3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 10 monclient(hunting): handle_auth_done global_id 4825 payload 824 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 1 -- 10.20.193.6:0/3579611582 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1200c99a0 msgr2=0x7fa1201f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1200c99a0 0x7fa1201f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 1 -- 10.20.193.6:0/3579611582 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa12006ff20 msgr2=0x7fa12014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa12006ff20 0x7fa12014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 10 monclient: _finish_hunting 0 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 1 monclient: found mon.noname-b 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.433+0000 7fa1265e5640 1 -- 10.20.193.6:0/3579611582 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1201f45a0 con 0x7fa12015a410 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.434+0000 7fa1265e5640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12015a410 0x7fa1201f5000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00cbd0 tx=0x7fa11c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.434+0000 7fa10e7fc640 10 client.?.objecter ms_handle_connect 0x7fa12015a410 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.434+0000 7fa10e7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 1 -- 10.20.193.6:0/3579611582 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00ad50 con 0x7fa12015a410 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:01.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: dump: 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:01.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: _finish_auth 0 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 1 -- 10.20.193.6:0/3579611582 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa11c003a70 con 0x7fa12015a410 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 1 -- 10.20.193.6:0/3579611582 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c003e40 con 0x7fa12015a410 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10e7fc640 10 monclient: dump: 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:01.449 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa10ffff640 4 set_mon_vals no callback set 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 5 monclient: authenticate success, global_id 4825 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 10 monclient: _renew_subs 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:01.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1200c2510 con 0x7fa12015a410 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 10 client.4825.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 10 monclient: _renew_subs 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1200c2ad0 con 0x7fa12015a410 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.435+0000 7fa12806f640 1 librados: init done 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.436+0000 7fa12806f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.436+0000 7fa12806f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.436+0000 7fa12806f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:01.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.436+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa120070300 con 0x7fa12015a410 2026-02-22T23:26:01.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.447+0000 7fa10e7fc640 1 -- 10.20.193.6:0/3579611582 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa11c0050d0 con 0x7fa12015a410 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.448+0000 7fa10e7fc640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0f0078e50 0x7fa0f007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.448+0000 7fa125de4640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0f0078e50 0x7fa0f007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.448+0000 7fa125de4640 10 monclient: get_auth_request con 0x7fa0f0078e50 auth_method 0 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.448+0000 7fa10e7fc640 1 -- 10.20.193.6:0/3579611582 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fa11c05e690 con 0x7fa12015a410 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.449+0000 7fa10e7fc640 10 client.4825.objecter ms_dispatch2 0x7fa120000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.449+0000 7fa10e7fc640 3 client.4825.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.449+0000 7fa10e7fc640 3 client.4825.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.449+0000 7fa10e7fc640 20 client.4825.objecter dump_active .. 0 homeless 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.449+0000 7fa10e7fc640 10 client.4825.objecter ms_handle_connect 0x7fa0f0078e50 2026-02-22T23:26:01.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.449+0000 7fa125de4640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0f0078e50 0x7fa0f007b310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1200c5380 tx=0x7fa110039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:01.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.550+0000 7fa10e7fc640 1 -- 10.20.193.6:0/3579611582 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa11c062650 con 0x7fa12015a410 2026-02-22T23:26:01.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.550+0000 7fa10e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:01.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.550+0000 7fa10e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:01.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.648+0000 7fa12806f640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:01.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.648+0000 7fa12806f640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:01.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.648+0000 7fa12806f640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:01.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.648+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa1201573f0 con 0x7fa12015a410 2026-02-22T23:26:01.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.651+0000 7fa10e7fc640 1 -- 10.20.193.6:0/3579611582 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fa11c066950 con 0x7fa12015a410 2026-02-22T23:26:01.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.651+0000 7fa10e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:01.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.651+0000 7fa10e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:01.652 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:01.652 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:01.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 10 librados: watch_flush enter 2026-02-22T23:26:01.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 10 librados: watch_flush exit 2026-02-22T23:26:01.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 20 client.4825.objecter shutdown clearing up homeless session... 2026-02-22T23:26:01.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 10 client.4825.objecter successfully canceled tick 2026-02-22T23:26:01.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0f0078e50 msgr2=0x7fa0f007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:01.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0f0078e50 0x7fa0f007b310 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1200c5380 tx=0x7fa110039040 comp rx=0 tx=0).stop 2026-02-22T23:26:01.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 10 monclient: shutdown 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12015a410 msgr2=0x7fa1201f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12015a410 0x7fa1201f5000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00cbd0 tx=0x7fa11c00a430 comp rx=0 tx=0).stop 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 shutdown_connections 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa0f0078e50 0x7fa0f007b310 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa12015a410 0x7fa1201f5000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1200c99a0 0x7fa1201f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 --2- 10.20.193.6:0/3579611582 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa12006ff20 0x7fa12014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 >> 10.20.193.6:0/3579611582 conn(0x7fa1200cd780 msgr2=0x7fa120157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 shutdown_connections 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 -- 10.20.193.6:0/3579611582 wait complete. 2026-02-22T23:26:01.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:01.652+0000 7fa12806f640 1 librados: shutdown 2026-02-22T23:26:04.663 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:04.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.729+0000 7f650141a640 10 monclient: get_monmap_and_config 2026-02-22T23:26:04.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.729+0000 7f650141a640 10 monclient: build_initial_monmap 2026-02-22T23:26:04.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.729+0000 7f650141a640 10 monclient: monmap: 2026-02-22T23:26:04.731 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:04.730465+0000 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:04.730465+0000 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:04.732 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:04.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.730+0000 7f650141a640 1 Processor -- start 2026-02-22T23:26:04.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.730+0000 7f650141a640 1 -- start start 2026-02-22T23:26:04.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.730+0000 7f650141a640 10 monclient: init 2026-02-22T23:26:04.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 10 monclient: _add_conns ranks=[2,1,0,3,5,4] 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f64fc0c90d0 addr v1:10.20.193.80:6789/0 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f64fc0c94b0 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f64fc074250 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc073300 con 0x7f64fc074250 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc159440 con 0x7f64fc0c94b0 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64fc159720 con 0x7f64fc0c90d0 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f650141a640 10 monclient(hunting): authenticate will time out at 1239.458144s 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f64fb7fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f64fc074250 0x7f64fc074680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:40512/0 (socket says 10.20.193.6:40512) 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.731+0000 7f64fb7fe640 1 -- 10.20.193.6:0/572698942 learned_addr learned my addr 10.20.193.6:0/572698942 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:04.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.732+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3395989480 0 0) 0x7f64fc073300 con 0x7f64fc074250 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.732+0000 7f64f9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.732+0000 7f64f9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.732+0000 7f64f9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.732+0000 7f64f9ffb640 10 monclient(hunting): my global_id is 4848 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.732+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e80058b0 con 0x7f64fc074250 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955315821 0 0) 0x7f64fc159720 con 0x7f64fc0c90d0 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): my global_id is 4958 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64fc073300 con 0x7f64fc0c90d0 2026-02-22T23:26:04.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134330372 0 0) 0x7f64fc159440 con 0x7f64fc0c94b0 2026-02-22T23:26:04.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:26:04.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:04.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 10 monclient(hunting): my global_id is 4828 2026-02-22T23:26:04.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.733+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64fc159720 con 0x7f64fc0c94b0 2026-02-22T23:26:04.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4220659721 0 0) 0x7f64e80058b0 con 0x7f64fc074250 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 >> v1:10.20.193.80:6789/0 conn(0x7f64fc0c90d0 legacy=0x7f64fc072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 >> v1:10.20.193.40:6789/0 conn(0x7f64fc0c94b0 legacy=0x7f64fc158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc073790 con 0x7f64fc074250 2026-02-22T23:26:04.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: _finish_auth 0 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f64ec003280 con 0x7f64fc074250 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: dump: 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:04.750 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 >> v1:10.20.193.6:6789/0 conn(0x7f64fc074250 legacy=0x7f64fc074680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64e8006270 0x7f64e8006b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:04.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): picked mon.a con 0x7f64e8006270 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64e8007550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): picked mon.c con 0x7f64e80070d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64e800bd40 0x7f64e800e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): picked mon.b con 0x7f64e800bd40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f64e800e800 con 0x7f64e8006270 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64e800e980 con 0x7f64e800bd40 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64e800eb80 con 0x7f64e80070d0 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:04.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.734+0000 7f64f9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.735+0000 7f64fa7fc640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64e8007550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.735+0000 7f64fa7fc640 10 monclient(hunting): get_auth_request con 0x7f64e80070d0 auth_method 0 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.735+0000 7f64fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.735+0000 7f64fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.735+0000 7f64fa7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.735+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.735+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64e800bd40 0x7f64e800e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64e8006270 0x7f64e8006b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 10 monclient(hunting): get_auth_request con 0x7f64e800bd40 auth_method 0 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:04.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 10 monclient(hunting): get_auth_request con 0x7f64e8006270 auth_method 0 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64faffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_done global_id 4848 payload 293 2026-02-22T23:26:04.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 1 -- 10.20.193.6:0/572698942 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64e800bd40 msgr2=0x7f64e800e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64e800bd40 0x7f64e800e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 1 -- 10.20.193.6:0/572698942 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64e8006270 msgr2=0x7f64e8006b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64e8006270 0x7f64e8006b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 1 monclient: found mon.c 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 1 -- 10.20.193.6:0/572698942 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e80058b0 con 0x7f64e80070d0 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 10 monclient: _finish_auth 0 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f64fb7fe640 1 -- 10.20.193.6:0/572698942 reap_dead start 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f650141a640 5 monclient: authenticate success, global_id 4848 2026-02-22T23:26:04.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.736+0000 7f650141a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.737+0000 7f64fa7fc640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64e8007550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f64e40031a0 tx=0x7f64e402eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c803945c25f7543f server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4002dc0 con 0x7f64e80070d0 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 10 monclient: dump: 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:04.756 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e40052d0 con 0x7f64e80070d0 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f650141a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 1 -- 10.20.193.6:0/572698942 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e402f740 con 0x7f64e80070d0 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f64f9ffb640 10 monclient: dump: 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:04.757 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f650141a640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f650141a640 4 set_mon_vals no callback set 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.739+0000 7f650141a640 10 monclient: shutdown 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 -- 10.20.193.6:0/572698942 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 msgr2=0x7f64e8007550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64e8007550 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f64e40031a0 tx=0x7f64e402eee0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 -- 10.20.193.6:0/572698942 shutdown_connections 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 --2- 10.20.193.6:0/572698942 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64e8007550 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 -- 10.20.193.6:0/572698942 >> 10.20.193.6:0/572698942 conn(0x7f64fc0cd780 msgr2=0x7f64fc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 -- 10.20.193.6:0/572698942 shutdown_connections 2026-02-22T23:26:04.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 -- 10.20.193.6:0/572698942 wait complete. 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 10 monclient: build_initial_monmap 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 10 monclient: monmap: 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 librados: starting msgr at 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 librados: starting objecter 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 Processor -- start 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 -- start start 2026-02-22T23:26:04.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 librados: setting wanted keys 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 1 librados: calling monclient init 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.740+0000 7f650141a640 10 monclient: init 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64fc15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): picked mon.noname-c con 0x7f64e80070d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64fc1f4b30 0x7f64fc0c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): picked mon.noname-a con 0x7f64fc1f4b30 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64fc0cc4e0 0x7f64fc0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): picked mon.noname-b con 0x7f64fc0cc4e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc073540 con 0x7f64fc1f4b30 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc073d50 con 0x7f64fc0cc4e0 2026-02-22T23:26:04.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc072bf0 con 0x7f64e80070d0 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f650141a640 10 monclient(hunting): authenticate will time out at 1239.468326s 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64fc1f4b30 0x7f64fc0c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64faffd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64fc15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fb7fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64fc0cc4e0 0x7f64fc0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64fc1f4b30 0x7f64fc0c05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:51856/0 (socket says 10.20.193.6:51856) 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 1 -- 10.20.193.6:0/2173661780 learned_addr learned my addr 10.20.193.6:0/2173661780 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 10 monclient(hunting): get_auth_request con 0x7f64fc1f4b30 auth_method 0 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64faffd640 10 monclient(hunting): get_auth_request con 0x7f64e80070d0 auth_method 0 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64faffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:04.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64faffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64faffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fb7fe640 10 monclient(hunting): get_auth_request con 0x7f64fc0cc4e0 auth_method 0 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.741+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.742+0000 7f64faffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.742+0000 7f64faffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.742+0000 7f64faffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.742+0000 7f64fb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.742+0000 7f64fb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.742+0000 7f64fb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 10 monclient(hunting): handle_auth_done global_id 4851 payload 824 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 1 -- 10.20.193.6:0/2173661780 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 msgr2=0x7f64fc15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64fc15b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 1 -- 10.20.193.6:0/2173661780 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64fc0cc4e0 msgr2=0x7f64fc0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64fc0cc4e0 0x7f64fc0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 1 monclient: found mon.noname-a 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.743+0000 7f64fa7fc640 1 -- 10.20.193.6:0/2173661780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc1f4fb0 con 0x7f64fc1f4b30 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.744+0000 7f64fa7fc640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64fc1f4b30 0x7f64fc0c05a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f64e40031a0 tx=0x7f64e4002ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.744+0000 7f64db7fe640 10 client.?.objecter ms_handle_connect 0x7f64fc1f4b30 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.744+0000 7f64db7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 1 -- 10.20.193.6:0/2173661780 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e40050d0 con 0x7f64fc1f4b30 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: dump: 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.763 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: _finish_auth 0 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 1 -- 10.20.193.6:0/2173661780 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e4044070 con 0x7f64fc1f4b30 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 1 -- 10.20.193.6:0/2173661780 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e402fb50 con 0x7f64fc1f4b30 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64db7fe640 10 monclient: dump: 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:04.764 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 5 monclient: authenticate success, global_id 4851 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f64f97fa640 4 set_mon_vals no callback set 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 monclient: _renew_subs 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc0ccea0 con 0x7f64fc1f4b30 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 client.4851.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 monclient: _renew_subs 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc15a370 con 0x7f64fc1f4b30 2026-02-22T23:26:04.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 1 librados: init done 2026-02-22T23:26:04.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:04.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:04.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:04.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.746+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc072fd0 con 0x7f64fc1f4b30 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.757+0000 7f64db7fe640 1 -- 10.20.193.6:0/2173661780 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f64e40036f0 con 0x7f64fc1f4b30 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.757+0000 7f64db7fe640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f64c8078e80 0x7f64c807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.758+0000 7f64faffd640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f64c8078e80 0x7f64c807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.758+0000 7f64faffd640 10 monclient: get_auth_request con 0x7f64c8078e80 auth_method 0 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.758+0000 7f64db7fe640 10 client.4851.objecter ms_handle_connect 0x7f64c8078e80 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.758+0000 7f64faffd640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f64c8078e80 0x7f64c807b340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64f0002910 tx=0x7f64f0007da0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.759+0000 7f64db7fe640 1 -- 10.20.193.6:0/2173661780 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f64e4005270 con 0x7f64fc1f4b30 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.759+0000 7f64db7fe640 10 client.4851.objecter ms_dispatch2 0x7f64fc000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.759+0000 7f64db7fe640 3 client.4851.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.759+0000 7f64db7fe640 3 client.4851.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:04.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.759+0000 7f64db7fe640 20 client.4851.objecter dump_active .. 0 homeless 2026-02-22T23:26:04.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.860+0000 7f64db7fe640 1 -- 10.20.193.6:0/2173661780 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f64fc1f4fb0 con 0x7f64fc1f4b30 2026-02-22T23:26:04.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.860+0000 7f64db7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:04.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.860+0000 7f64db7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:04.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.960+0000 7f650141a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:04.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.960+0000 7f650141a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:04.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.960+0000 7f650141a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:04.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.960+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f64fc15a650 con 0x7f64fc1f4b30 2026-02-22T23:26:04.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.962+0000 7f64db7fe640 1 -- 10.20.193.6:0/2173661780 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f64e40873b0 con 0x7f64fc1f4b30 2026-02-22T23:26:04.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.962+0000 7f64db7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:04.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.962+0000 7f64db7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:04.964 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:04.964 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:04.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 10 librados: watch_flush enter 2026-02-22T23:26:04.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 10 librados: watch_flush exit 2026-02-22T23:26:04.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 20 client.4851.objecter shutdown clearing up homeless session... 2026-02-22T23:26:04.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 10 client.4851.objecter successfully canceled tick 2026-02-22T23:26:04.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f64c8078e80 msgr2=0x7f64c807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f64c8078e80 0x7f64c807b340 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64f0002910 tx=0x7f64f0007da0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 10 monclient: shutdown 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64fc1f4b30 msgr2=0x7f64fc0c05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64fc1f4b30 0x7f64fc0c05a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f64e40031a0 tx=0x7f64e4002ac0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 shutdown_connections 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f64c8078e80 0x7f64c807b340 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f64fc0cc4e0 0x7f64fc0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f64fc1f4b30 0x7f64fc0c05a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 --2- 10.20.193.6:0/2173661780 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f64e80070d0 0x7f64fc15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 >> 10.20.193.6:0/2173661780 conn(0x7f64fc0cd780 msgr2=0x7f64fc0c8710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:04.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 shutdown_connections 2026-02-22T23:26:04.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 -- 10.20.193.6:0/2173661780 wait complete. 2026-02-22T23:26:04.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:04.963+0000 7f650141a640 1 librados: shutdown 2026-02-22T23:26:07.974 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.040+0000 7f7d388a9640 10 monclient: get_monmap_and_config 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.040+0000 7f7d388a9640 10 monclient: build_initial_monmap 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.040+0000 7f7d388a9640 10 monclient: monmap: 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:08.041819+0000 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:08.041819+0000 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:08.043 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.040+0000 7f7d388a9640 1 Processor -- start 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.040+0000 7f7d388a9640 1 -- start start 2026-02-22T23:26:08.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.040+0000 7f7d388a9640 10 monclient: init 2026-02-22T23:26:08.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:08.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient: _add_conns ranks=[1,0,3,2,5,4] 2026-02-22T23:26:08.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7d3006ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:08.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7d300c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:08.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f7d3015a5e0 0x7f7d30070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:08.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient(hunting): picked mon.noname-a con 0x7f7d3015a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d300c7e90 con 0x7f7d300c99a0 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d300c8170 con 0x7f7d3006ff20 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7d30159a40 con 0x7f7d3015a5e0 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d388a9640 10 monclient(hunting): authenticate will time out at 1242.769185s 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d3661e640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f7d3006ff20 0x7f7d30159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:57110/0 (socket says 10.20.193.6:57110) 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d3661e640 1 -- 10.20.193.6:0/1409056234 learned_addr learned my addr 10.20.193.6:0/1409056234 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.041+0000 7f7d36e1f640 1 --2- 10.20.193.6:0/1409056234 >> v2:10.20.193.6:3300/0 conn(0x7f7d3015a5e0 0x7f7d30070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.042+0000 7f7d36e1f640 10 monclient(hunting): get_auth_request con 0x7f7d3015a5e0 auth_method 0 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.042+0000 7f7d36e1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:08.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.042+0000 7f7d36e1f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.042+0000 7f7d36e1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.042+0000 7f7d36e1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.042+0000 7f7d36e1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.042+0000 7f7d36e1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.043+0000 7f7d3561c640 1 -- 10.20.193.6:0/1409056234 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4195040030 0 0) 0x7f7d300c8170 con 0x7f7d3006ff20 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.043+0000 7f7d3561c640 10 monclient(hunting): _negotiate 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.043+0000 7f7d3561c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.043+0000 7f7d3561c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.043+0000 7f7d3561c640 10 monclient(hunting): my global_id is 4834 2026-02-22T23:26:08.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.043+0000 7f7d3561c640 1 -- 10.20.193.6:0/1409056234 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d20003620 con 0x7f7d3006ff20 2026-02-22T23:26:08.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 10 monclient(hunting): handle_auth_done global_id 4854 payload 274 2026-02-22T23:26:08.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 1 -- 10.20.193.6:0/1409056234 >> v1:10.20.193.40:6789/0 conn(0x7f7d3006ff20 legacy=0x7f7d30159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:08.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 1 -- 10.20.193.6:0/1409056234 >> v1:10.20.193.6:6789/0 conn(0x7f7d300c99a0 legacy=0x7f7d3014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:08.049 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 10 monclient: _finish_hunting 0 2026-02-22T23:26:08.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 1 monclient: found mon.noname-a 2026-02-22T23:26:08.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:08.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:08.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.044+0000 7f7d36e1f640 1 -- 10.20.193.6:0/1409056234 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d300c8450 con 0x7f7d3015a5e0 2026-02-22T23:26:08.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.045+0000 7f7d36e1f640 1 --2- 10.20.193.6:0/1409056234 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3015a5e0 0x7f7d30070300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009dc0 tx=0x7f7d2402f280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855105627f2f4a44 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 1 -- 10.20.193.6:0/1409056234 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2403c070 con 0x7f7d3015a5e0 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: dump: 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.056 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: _finish_auth 0 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: _check_auth_tickets 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 1 -- 10.20.193.6:0/1409056234 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d24037440 con 0x7f7d3015a5e0 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 1 -- 10.20.193.6:0/1409056234 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24037810 con 0x7f7d3015a5e0 2026-02-22T23:26:08.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d3561c640 10 monclient: dump: 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:08.058 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d388a9640 5 monclient: authenticate success, global_id 4854 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d388a9640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.047+0000 7f7d388a9640 4 set_mon_vals no callback set 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 10 monclient: shutdown 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 -- 10.20.193.6:0/1409056234 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3015a5e0 msgr2=0x7f7d30070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1409056234 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3015a5e0 0x7f7d30070300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009dc0 tx=0x7f7d2402f280 comp rx=0 tx=0).stop 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 -- 10.20.193.6:0/1409056234 shutdown_connections 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1409056234 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3015a5e0 0x7f7d30070300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 -- 10.20.193.6:0/1409056234 >> 10.20.193.6:0/1409056234 conn(0x7f7d300cd780 msgr2=0x7f7d300cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 -- 10.20.193.6:0/1409056234 shutdown_connections 2026-02-22T23:26:08.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 -- 10.20.193.6:0/1409056234 wait complete. 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 10 monclient: build_initial_monmap 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 10 monclient: monmap: 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:08.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 librados: starting msgr at 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 librados: starting objecter 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 Processor -- start 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 -- start start 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 librados: setting wanted keys 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 1 librados: calling monclient init 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.048+0000 7f7d388a9640 10 monclient: init 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3006ff20 0x7f7d30147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): picked mon.noname-a con 0x7f7d3006ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d300c99a0 0x7f7d300c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): picked mon.noname-c con 0x7f7d300c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:08.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d3015a5e0 0x7f7d300c1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): picked mon.noname-b con 0x7f7d3015a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30150520 con 0x7f7d3006ff20 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300c8a10 con 0x7f7d3015a5e0 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301506a0 con 0x7f7d300c99a0 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d388a9640 10 monclient(hunting): authenticate will time out at 1242.777341s 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d3661e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3006ff20 0x7f7d30147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d36e1f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d3015a5e0 0x7f7d300c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d3661e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3006ff20 0x7f7d30147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47662/0 (socket says 10.20.193.6:47662) 2026-02-22T23:26:08.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d3661e640 1 -- 10.20.193.6:0/1990469976 learned_addr learned my addr 10.20.193.6:0/1990469976 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d3661e640 10 monclient(hunting): get_auth_request con 0x7f7d3006ff20 auth_method 0 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d3661e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d3661e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d3661e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d35e1d640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d300c99a0 0x7f7d300c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d36e1f640 10 monclient(hunting): get_auth_request con 0x7f7d3015a5e0 auth_method 0 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d36e1f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d36e1f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.049+0000 7f7d36e1f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d3661e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d3661e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:08.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d3661e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d35e1d640 10 monclient(hunting): get_auth_request con 0x7f7d300c99a0 auth_method 0 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d35e1d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d35e1d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d35e1d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d36e1f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d36e1f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d36e1f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d35e1d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d35e1d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.050+0000 7f7d35e1d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 10 monclient(hunting): handle_auth_done global_id 4860 payload 824 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 1 -- 10.20.193.6:0/1990469976 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d300c99a0 msgr2=0x7f7d300c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:08.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d300c99a0 0x7f7d300c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 1 -- 10.20.193.6:0/1990469976 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d3015a5e0 msgr2=0x7f7d300c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d3015a5e0 0x7f7d300c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 10 monclient: _finish_hunting 0 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 1 monclient: found mon.noname-a 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.051+0000 7f7d3661e640 1 -- 10.20.193.6:0/1990469976 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d30148600 con 0x7f7d3006ff20 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.052+0000 7f7d3661e640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3006ff20 0x7f7d30147c50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c700 tx=0x7f7d2800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.052+0000 7f7d1affd640 10 client.?.objecter ms_handle_connect 0x7f7d3006ff20 2026-02-22T23:26:08.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.052+0000 7f7d1affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:08.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 1 -- 10.20.193.6:0/1990469976 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28017070 con 0x7f7d3006ff20 2026-02-22T23:26:08.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:08.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:08.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: dump: 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: _finish_auth 0 2026-02-22T23:26:08.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: _check_auth_tickets 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 1 -- 10.20.193.6:0/1990469976 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d28005bf0 con 0x7f7d3006ff20 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 5 monclient: authenticate success, global_id 4860 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 monclient: _renew_subs 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d301488e0 con 0x7f7d3006ff20 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 1 -- 10.20.193.6:0/1990469976 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28007070 con 0x7f7d3006ff20 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d34e1b640 4 set_mon_vals no callback set 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d1affd640 10 monclient: dump: 2026-02-22T23:26:08.069 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 client.4860.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 monclient: _renew_subs 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:08.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d30148ea0 con 0x7f7d3006ff20 2026-02-22T23:26:08.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 1 librados: init done 2026-02-22T23:26:08.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:08.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:08.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:08.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.054+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d30070300 con 0x7f7d3006ff20 2026-02-22T23:26:08.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.066+0000 7f7d1affd640 1 -- 10.20.193.6:0/1990469976 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7d28007210 con 0x7f7d3006ff20 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.066+0000 7f7d1affd640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d00078e50 0x7f7d0007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.067+0000 7f7d35e1d640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d00078e50 0x7f7d0007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.067+0000 7f7d35e1d640 10 monclient: get_auth_request con 0x7f7d00078e50 auth_method 0 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.067+0000 7f7d1affd640 10 client.4860.objecter ms_handle_connect 0x7f7d00078e50 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.067+0000 7f7d35e1d640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d00078e50 0x7f7d0007b310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c002410 tx=0x7f7d1c007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.068+0000 7f7d1affd640 1 -- 10.20.193.6:0/1990469976 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f7d2805d4c0 con 0x7f7d3006ff20 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.068+0000 7f7d1affd640 10 client.4860.objecter ms_dispatch2 0x7f7d30000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.068+0000 7f7d1affd640 3 client.4860.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.068+0000 7f7d1affd640 3 client.4860.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:08.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.069+0000 7f7d1affd640 20 client.4860.objecter dump_active .. 0 homeless 2026-02-22T23:26:08.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.166+0000 7f7d1affd640 1 -- 10.20.193.6:0/1990469976 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7d28021020 con 0x7f7d3006ff20 2026-02-22T23:26:08.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.166+0000 7f7d1affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:08.169 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.166+0000 7f7d1affd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:08.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.270+0000 7f7d388a9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:08.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.270+0000 7f7d388a9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:08.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.270+0000 7f7d388a9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:08.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.270+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7d300c4a10 con 0x7f7d3006ff20 2026-02-22T23:26:08.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.272+0000 7f7d1affd640 1 -- 10.20.193.6:0/1990469976 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7d28061480 con 0x7f7d3006ff20 2026-02-22T23:26:08.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.272+0000 7f7d1affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:08.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.272+0000 7f7d1affd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:08.275 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:08.275 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:08.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 10 librados: watch_flush enter 2026-02-22T23:26:08.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 10 librados: watch_flush exit 2026-02-22T23:26:08.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 20 client.4860.objecter shutdown clearing up homeless session... 2026-02-22T23:26:08.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 10 client.4860.objecter successfully canceled tick 2026-02-22T23:26:08.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d00078e50 msgr2=0x7f7d0007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:08.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d00078e50 0x7f7d0007b310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c002410 tx=0x7f7d1c007ef0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 10 monclient: shutdown 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3006ff20 msgr2=0x7f7d30147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.273+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3006ff20 0x7f7d30147c50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c700 tx=0x7f7d2800cbd0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 shutdown_connections 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7d00078e50 0x7f7d0007b310 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7d3015a5e0 0x7f7d300c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7d300c99a0 0x7f7d300c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 --2- 10.20.193.6:0/1990469976 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7d3006ff20 0x7f7d30147c50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 >> 10.20.193.6:0/1990469976 conn(0x7f7d300cd780 msgr2=0x7f7d3014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 shutdown_connections 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 -- 10.20.193.6:0/1990469976 wait complete. 2026-02-22T23:26:08.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:08.274+0000 7f7d388a9640 1 librados: shutdown 2026-02-22T23:26:11.286 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.352+0000 7f13d85c3640 10 monclient: get_monmap_and_config 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.352+0000 7f13d85c3640 10 monclient: build_initial_monmap 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.352+0000 7f13d85c3640 10 monclient: monmap: 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:11.354219+0000 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:11.354219+0000 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 Processor -- start 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 -- start start 2026-02-22T23:26:11.355 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient: init 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient: _add_conns ranks=[5,4,3,1,0,2] 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f13d0155ef0 0x7f13d015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): picked mon.noname-c con 0x7f13d0155ef0 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f13d01562d0 0x7f13d00cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): picked mon.noname-b con 0x7f13d01562d0 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f13d00cde80 0x7f13d00c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): picked mon.noname-a con 0x7f13d00cde80 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f13d015cdb0 con 0x7f13d00cde80 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f13d00d2730 con 0x7f13d01562d0 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f13d00ce3b0 con 0x7f13d0155ef0 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:11.356 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.353+0000 7f13d85c3640 10 monclient(hunting): authenticate will time out at 1246.081868s 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f13d00cde80 0x7f13d00c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f13d0155ef0 0x7f13d015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f13d01562d0 0x7f13d00cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f13d00cde80 0x7f13d00c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47670/0 (socket says 10.20.193.6:47670) 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 1 -- 10.20.193.6:0/452373063 learned_addr learned my addr 10.20.193.6:0/452373063 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 10 monclient(hunting): get_auth_request con 0x7f13d00cde80 auth_method 0 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 10 monclient(hunting): get_auth_request con 0x7f13d01562d0 auth_method 0 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 10 monclient(hunting): get_auth_request con 0x7f13d0155ef0 auth_method 0 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d5b37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.354+0000 7f13d6338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 10 monclient(hunting): handle_auth_done global_id 4840 payload 274 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 1 -- 10.20.193.6:0/452373063 >> v2:10.20.193.80:3300/0 conn(0x7f13d0155ef0 msgr2=0x7f13d015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:11.357 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 1 --2- 10.20.193.6:0/452373063 >> v2:10.20.193.80:3300/0 conn(0x7f13d0155ef0 0x7f13d015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 1 -- 10.20.193.6:0/452373063 >> v2:10.20.193.6:3300/0 conn(0x7f13d00cde80 msgr2=0x7f13d00c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 1 --2- 10.20.193.6:0/452373063 >> v2:10.20.193.6:3300/0 conn(0x7f13d00cde80 0x7f13d00c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 10 monclient: _finish_hunting 0 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 1 monclient: found mon.noname-b 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.355+0000 7f13d5b37640 1 -- 10.20.193.6:0/452373063 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d00ce690 con 0x7f13d01562d0 2026-02-22T23:26:11.358 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.356+0000 7f13d5b37640 1 --2- 10.20.193.6:0/452373063 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d01562d0 0x7f13d00cb620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13d008e920 tx=0x7f13c402ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c7ebb445683817 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 1 -- 10.20.193.6:0/452373063 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c403c070 con 0x7f13d01562d0 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: dump: 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: _finish_auth 0 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: _check_auth_tickets 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 1 -- 10.20.193.6:0/452373063 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c402fb00 con 0x7f13d01562d0 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 5 monclient: authenticate success, global_id 4840 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 4 set_mon_vals no callback set 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 1 -- 10.20.193.6:0/452373063 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4037440 con 0x7f13d01562d0 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d5336640 10 monclient: dump: 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.360 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 10 monclient: shutdown 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 -- 10.20.193.6:0/452373063 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d01562d0 msgr2=0x7f13d00cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 --2- 10.20.193.6:0/452373063 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d01562d0 0x7f13d00cb620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13d008e920 tx=0x7f13c402ee80 comp rx=0 tx=0).stop 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 -- 10.20.193.6:0/452373063 shutdown_connections 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 --2- 10.20.193.6:0/452373063 >> v2:10.20.193.6:3300/0 conn(0x7f13d00cde80 0x7f13d00c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 --2- 10.20.193.6:0/452373063 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d01562d0 0x7f13d00cb620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 --2- 10.20.193.6:0/452373063 >> v2:10.20.193.80:3300/0 conn(0x7f13d0155ef0 0x7f13d015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 -- 10.20.193.6:0/452373063 >> 10.20.193.6:0/452373063 conn(0x7f13d006d8c0 msgr2=0x7f13d006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 -- 10.20.193.6:0/452373063 shutdown_connections 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.358+0000 7f13d85c3640 1 -- 10.20.193.6:0/452373063 wait complete. 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient: build_initial_monmap 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient: monmap: 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:11.361 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 librados: starting msgr at 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 librados: starting objecter 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 Processor -- start 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 -- start start 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 librados: setting wanted keys 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 librados: calling monclient init 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient: init 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:11.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d00cde80 0x7f13d01f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): picked mon.noname-b con 0x7f13d00cde80 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f13d0155ef0 0x7f13d0073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): picked mon.noname-a con 0x7f13d0155ef0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f13d01562d0 0x7f13d0149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): picked mon.noname-c con 0x7f13d01562d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f13d00cef30 con 0x7f13d0155ef0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f13d00cec50 con 0x7f13d00cde80 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13d00cbb60 con 0x7f13d01562d0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.359+0000 7f13d85c3640 10 monclient(hunting): authenticate will time out at 1246.087916s 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f13d0155ef0 0x7f13d0073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f13d01562d0 0x7f13d0149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d00cde80 0x7f13d01f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f13d0155ef0 0x7f13d0073230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47672/0 (socket says 10.20.193.6:47672) 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 1 -- 10.20.193.6:0/472491854 learned_addr learned my addr 10.20.193.6:0/472491854 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 10 monclient(hunting): get_auth_request con 0x7f13d0155ef0 auth_method 0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 10 monclient(hunting): get_auth_request con 0x7f13d01562d0 auth_method 0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 10 monclient(hunting): get_auth_request con 0x7f13d00cde80 auth_method 0 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d5b37640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6338640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:11.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.360+0000 7f13d6b39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.361+0000 7f13d6338640 10 monclient(hunting): handle_auth_done global_id 4843 payload 824 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.361+0000 7f13d6338640 1 -- 10.20.193.6:0/472491854 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f13d01562d0 msgr2=0x7f13d0149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.361+0000 7f13d6338640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f13d01562d0 0x7f13d0149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.361+0000 7f13d6338640 1 -- 10.20.193.6:0/472491854 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f13d0155ef0 msgr2=0x7f13d0073230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.361+0000 7f13d6338640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f13d0155ef0 0x7f13d0073230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.361+0000 7f13d6338640 10 monclient: _finish_hunting 0 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.361+0000 7f13d6338640 1 monclient: found mon.noname-b 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.362+0000 7f13d6338640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.362+0000 7f13d6338640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:11.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.362+0000 7f13d6338640 1 -- 10.20.193.6:0/472491854 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d013f460 con 0x7f13d00cde80 2026-02-22T23:26:11.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.362+0000 7f13d6338640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d00cde80 0x7f13d01f7c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e870 tx=0x7f13c000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:11.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.363+0000 7f13b6ffd640 10 client.?.objecter ms_handle_connect 0x7f13d00cde80 2026-02-22T23:26:11.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.363+0000 7f13b6ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 1 -- 10.20.193.6:0/472491854 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0017070 con 0x7f13d00cde80 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: dump: 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 1 -- 10.20.193.6:0/472491854 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c000c940 con 0x7f13d00cde80 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 1 -- 10.20.193.6:0/472491854 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c000cd10 con 0x7f13d00cde80 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 5 monclient: authenticate success, global_id 4843 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 10 monclient: _renew_subs 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d4b35640 4 set_mon_vals no callback set 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d015beb0 con 0x7f13d00cde80 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13b6ffd640 10 monclient: dump: 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:11.367 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 10 client.4843.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 10 monclient: _renew_subs 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d015ae10 con 0x7f13d00cde80 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.364+0000 7f13d85c3640 1 librados: init done 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.365+0000 7f13d85c3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.365+0000 7f13d85c3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.365+0000 7f13d85c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:11.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.365+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d015c870 con 0x7f13d00cde80 2026-02-22T23:26:11.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.376+0000 7f13b6ffd640 1 -- 10.20.193.6:0/472491854 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f13c0024060 con 0x7f13d00cde80 2026-02-22T23:26:11.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13b6ffd640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f13a0078e30 0x7f13a007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:11.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13d5b37640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f13a0078e30 0x7f13a007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:11.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13d5b37640 10 monclient: get_auth_request con 0x7f13a0078e30 auth_method 0 2026-02-22T23:26:11.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13b6ffd640 1 -- 10.20.193.6:0/472491854 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f13c005dfb0 con 0x7f13d00cde80 2026-02-22T23:26:11.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13b6ffd640 10 client.4843.objecter ms_dispatch2 0x7f13d0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:11.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13b6ffd640 3 client.4843.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:11.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13b6ffd640 3 client.4843.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:11.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13b6ffd640 20 client.4843.objecter dump_active .. 0 homeless 2026-02-22T23:26:11.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13d5b37640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f13a0078e30 0x7f13a007b2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13c402eec0 tx=0x7f13c4033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:11.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.377+0000 7f13b6ffd640 10 client.4843.objecter ms_handle_connect 0x7f13a0078e30 2026-02-22T23:26:11.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.477+0000 7f13b6ffd640 1 -- 10.20.193.6:0/472491854 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f13d013f460 con 0x7f13d00cde80 2026-02-22T23:26:11.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.477+0000 7f13b6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:11.479 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.477+0000 7f13b6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:11.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.577+0000 7f13d85c3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:11.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.577+0000 7f13d85c3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:11.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.577+0000 7f13d85c3640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:11.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.577+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f13d0159810 con 0x7f13d00cde80 2026-02-22T23:26:11.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13b6ffd640 1 -- 10.20.193.6:0/472491854 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f13c0061f70 con 0x7f13d00cde80 2026-02-22T23:26:11.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13b6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:11.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13b6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:11.582 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:11.582 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 10 librados: watch_flush enter 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 10 librados: watch_flush exit 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 20 client.4843.objecter shutdown clearing up homeless session... 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 10 client.4843.objecter successfully canceled tick 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f13a0078e30 msgr2=0x7f13a007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f13a0078e30 0x7f13a007b2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13c402eec0 tx=0x7f13c4033000 comp rx=0 tx=0).stop 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 10 monclient: shutdown 2026-02-22T23:26:11.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d00cde80 msgr2=0x7f13d01f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d00cde80 0x7f13d01f7c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13c000e870 tx=0x7f13c000ed40 comp rx=0 tx=0).stop 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 shutdown_connections 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f13a0078e30 0x7f13a007b2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f13d01562d0 0x7f13d0149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f13d0155ef0 0x7f13d0073230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 --2- 10.20.193.6:0/472491854 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f13d00cde80 0x7f13d01f7c20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 >> 10.20.193.6:0/472491854 conn(0x7f13d006d8c0 msgr2=0x7f13d014b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 shutdown_connections 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 -- 10.20.193.6:0/472491854 wait complete. 2026-02-22T23:26:11.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:11.580+0000 7f13d85c3640 1 librados: shutdown 2026-02-22T23:26:13.216 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:04:57.466 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.217 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:04:57.467 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.225 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:04:57.475 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.244 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:04:57.455 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.245 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:04:57.456 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.249 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:04:57.495 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.250 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:04:57.496 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.252 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:04:57.464 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:13.257 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:04:57.504 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:14.593 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:14.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.658+0000 7f1259b41640 10 monclient: get_monmap_and_config 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.658+0000 7f1259b41640 10 monclient: build_initial_monmap 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.658+0000 7f1259b41640 10 monclient: monmap: 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:14.659937+0000 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:14.659937+0000 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:14.661 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:14.662 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:14.662 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:14.662 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:14.662 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:14.662 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:14.662 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:14.662 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:14.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.658+0000 7f1259b41640 1 Processor -- start 2026-02-22T23:26:14.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.658+0000 7f1259b41640 1 -- start start 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.658+0000 7f1259b41640 10 monclient: init 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient: _add_conns ranks=[3,4,0,2,1,5] 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1254155d50 0x7f125415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient(hunting): picked mon.noname-a con 0x7f1254155d50 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f12540ccb50 0x7f125415d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient(hunting): picked mon.noname-b con 0x7f12540ccb50 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f125415d760 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:14.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540692c0 con 0x7f125415d760 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f125406c9a0 con 0x7f1254155d50 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f125406cc80 con 0x7f12540ccb50 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1259b41640 10 monclient(hunting): authenticate will time out at 1249.387446s 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f124bfff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f12540ccb50 0x7f125415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.659+0000 7f1258b3f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1254155d50 0x7f125415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:14.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1259340640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f125415d760 0x7f125415f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:60448/0 (socket says 10.20.193.6:60448) 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1259340640 1 -- 10.20.193.6:0/2758210414 learned_addr learned my addr 10.20.193.6:0/2758210414 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f124bfff640 10 monclient(hunting): get_auth_request con 0x7f12540ccb50 auth_method 0 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f124bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f124bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f124bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1258b3f640 10 monclient(hunting): get_auth_request con 0x7f1254155d50 auth_method 0 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1258b3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1258b3f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1258b3f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f124bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f124bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:14.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f124bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1258b3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1258b3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.660+0000 7f1258b3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 10 monclient(hunting): handle_auth_done global_id 4846 payload 274 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 1 -- 10.20.193.6:0/2758210414 >> v2:10.20.193.6:3300/0 conn(0x7f1254155d50 msgr2=0x7f125415c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 1 --2- 10.20.193.6:0/2758210414 >> v2:10.20.193.6:3300/0 conn(0x7f1254155d50 0x7f125415c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 1 -- 10.20.193.6:0/2758210414 >> v1:10.20.193.6:6789/0 conn(0x7f125415d760 legacy=0x7f125415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 1 monclient: found mon.noname-b 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.661+0000 7f124bfff640 1 -- 10.20.193.6:0/2758210414 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125406cf60 con 0x7f12540ccb50 2026-02-22T23:26:14.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.662+0000 7f124bfff640 1 --2- 10.20.193.6:0/2758210414 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f12540ccb50 0x7f125415d220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1254071b10 tx=0x7f123c02ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec17e45678c71ca server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:14.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 1 -- 10.20.193.6:0/2758210414 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c03c070 con 0x7f12540ccb50 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: dump: 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:14.678 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 1 -- 10.20.193.6:0/2758210414 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c02faf0 con 0x7f12540ccb50 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 5 monclient: authenticate success, global_id 4846 2026-02-22T23:26:14.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 1 -- 10.20.193.6:0/2758210414 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c037440 con 0x7f12540ccb50 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 4 set_mon_vals no callback set 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f124b7fe640 10 monclient: dump: 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:14.680 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 10 monclient: shutdown 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 -- 10.20.193.6:0/2758210414 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f12540ccb50 msgr2=0x7f125415d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 --2- 10.20.193.6:0/2758210414 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f12540ccb50 0x7f125415d220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1254071b10 tx=0x7f123c02ede0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 -- 10.20.193.6:0/2758210414 shutdown_connections 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 --2- 10.20.193.6:0/2758210414 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f12540ccb50 0x7f125415d220 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 --2- 10.20.193.6:0/2758210414 >> v2:10.20.193.6:3300/0 conn(0x7f1254155d50 0x7f125415c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 -- 10.20.193.6:0/2758210414 >> 10.20.193.6:0/2758210414 conn(0x7f12540cc030 msgr2=0x7f12540cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:14.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 -- 10.20.193.6:0/2758210414 shutdown_connections 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.664+0000 7f1259b41640 1 -- 10.20.193.6:0/2758210414 wait complete. 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 10 monclient: build_initial_monmap 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 10 monclient: monmap: 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:14.682 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 1 librados: starting msgr at 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 1 librados: starting objecter 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 1 Processor -- start 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 1 -- start start 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 1 librados: setting wanted keys 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 1 librados: calling monclient init 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.665+0000 7f1259b41640 10 monclient: init 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f12540ccb50 0x7f12540c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): picked mon.noname-a con 0x7f12540ccb50 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1254155d50 0x7f12540ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:14.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): picked mon.noname-b con 0x7f1254155d50 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f125415d760 0x7f125415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): picked mon.noname-c con 0x7f125415d760 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1254156130 con 0x7f12540ccb50 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f125406d520 con 0x7f1254155d50 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f12540d03c0 con 0x7f125415d760 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:14.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259b41640 10 monclient(hunting): authenticate will time out at 1249.394083s 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f12540ccb50 0x7f12540c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1254155d50 0x7f12540ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f12540ccb50 0x7f12540c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47682/0 (socket says 10.20.193.6:47682) 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 1 -- 10.20.193.6:0/2712118804 learned_addr learned my addr 10.20.193.6:0/2712118804 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 10 monclient(hunting): get_auth_request con 0x7f12540ccb50 auth_method 0 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 10 monclient(hunting): get_auth_request con 0x7f1254155d50 auth_method 0 2026-02-22T23:26:14.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1258b3f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259340640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f125415d760 0x7f125415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f124bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259340640 10 monclient(hunting): get_auth_request con 0x7f125415d760 auth_method 0 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259340640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259340640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:14.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.666+0000 7f1259340640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.667+0000 7f1259340640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.667+0000 7f1259340640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.667+0000 7f1259340640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 10 monclient(hunting): handle_auth_done global_id 4875 payload 824 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 1 -- 10.20.193.6:0/2712118804 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f125415d760 msgr2=0x7f125415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f125415d760 0x7f125415b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 1 -- 10.20.193.6:0/2712118804 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1254155d50 msgr2=0x7f12540ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1254155d50 0x7f12540ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 10 monclient: _finish_hunting 0 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 1 monclient: found mon.noname-a 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:14.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.668+0000 7f1258b3f640 1 -- 10.20.193.6:0/2712118804 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540c7e40 con 0x7f12540ccb50 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.669+0000 7f1258b3f640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f12540ccb50 0x7f12540c6e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f124400e890 tx=0x7f124400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.669+0000 7f12497fa640 10 client.?.objecter ms_handle_connect 0x7f12540ccb50 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.669+0000 7f12497fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 1 -- 10.20.193.6:0/2712118804 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244017070 con 0x7f12540ccb50 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: dump: 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.688 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: _finish_auth 0 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: _check_auth_tickets 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 1 -- 10.20.193.6:0/2712118804 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124400c940 con 0x7f12540ccb50 2026-02-22T23:26:14.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 1 -- 10.20.193.6:0/2712118804 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124400cd30 con 0x7f12540ccb50 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f12497fa640 10 monclient: dump: 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:14.690 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f124affd640 4 set_mon_vals no callback set 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 5 monclient: authenticate success, global_id 4875 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 monclient: _renew_subs 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12540c75a0 con 0x7f12540ccb50 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 client.4875.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 monclient: _renew_subs 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12540c7b60 con 0x7f12540ccb50 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 1 librados: init done 2026-02-22T23:26:14.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:14.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:14.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:14.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.671+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125415d220 con 0x7f12540ccb50 2026-02-22T23:26:14.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.683+0000 7f12497fa640 1 -- 10.20.193.6:0/2712118804 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1244024060 con 0x7f12540ccb50 2026-02-22T23:26:14.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.683+0000 7f12497fa640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1224078e00 0x7f122407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:14.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.683+0000 7f124bfff640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1224078e00 0x7f122407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:14.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.683+0000 7f124bfff640 10 monclient: get_auth_request con 0x7f1224078e00 auth_method 0 2026-02-22T23:26:14.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.684+0000 7f12497fa640 10 client.4875.objecter ms_handle_connect 0x7f1224078e00 2026-02-22T23:26:14.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.684+0000 7f124bfff640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1224078e00 0x7f122407b2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f123c039000 tx=0x7f123c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:14.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.685+0000 7f12497fa640 1 -- 10.20.193.6:0/2712118804 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f124405e080 con 0x7f12540ccb50 2026-02-22T23:26:14.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.685+0000 7f12497fa640 10 client.4875.objecter ms_dispatch2 0x7f1254000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:14.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.685+0000 7f12497fa640 3 client.4875.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:14.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.685+0000 7f12497fa640 3 client.4875.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:14.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.685+0000 7f12497fa640 20 client.4875.objecter dump_active .. 0 homeless 2026-02-22T23:26:14.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.785+0000 7f12497fa640 1 -- 10.20.193.6:0/2712118804 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1244062040 con 0x7f12540ccb50 2026-02-22T23:26:14.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.785+0000 7f12497fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:14.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.785+0000 7f12497fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:14.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.888+0000 7f1259b41640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:14.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.888+0000 7f1259b41640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:14.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.888+0000 7f1259b41640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:14.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.888+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f125415ecb0 con 0x7f12540ccb50 2026-02-22T23:26:14.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.891+0000 7f12497fa640 1 -- 10.20.193.6:0/2712118804 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1244066340 con 0x7f12540ccb50 2026-02-22T23:26:14.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.891+0000 7f12497fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:14.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.891+0000 7f12497fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:14.894 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:14.894 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 10 librados: watch_flush enter 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 10 librados: watch_flush exit 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 20 client.4875.objecter shutdown clearing up homeless session... 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 10 client.4875.objecter successfully canceled tick 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1224078e00 msgr2=0x7f122407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1224078e00 0x7f122407b2c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f123c039000 tx=0x7f123c033000 comp rx=0 tx=0).stop 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 10 monclient: shutdown 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:14.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f12540ccb50 msgr2=0x7f12540c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f12540ccb50 0x7f12540c6e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f124400e890 tx=0x7f124400ed60 comp rx=0 tx=0).stop 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 shutdown_connections 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1224078e00 0x7f122407b2c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f125415d760 0x7f125415b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1254155d50 0x7f12540ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 --2- 10.20.193.6:0/2712118804 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f12540ccb50 0x7f12540c6e40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 >> 10.20.193.6:0/2712118804 conn(0x7f12540cc030 msgr2=0x7f12540cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 shutdown_connections 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 -- 10.20.193.6:0/2712118804 wait complete. 2026-02-22T23:26:14.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:14.892+0000 7f1259b41640 1 librados: shutdown 2026-02-22T23:26:17.905 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:17.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.971+0000 7f75fe1bc640 10 monclient: get_monmap_and_config 2026-02-22T23:26:17.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.971+0000 7f75fe1bc640 10 monclient: build_initial_monmap 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.971+0000 7f75fe1bc640 10 monclient: monmap: 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:17.973193+0000 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:17.973193+0000 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:17.975 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:17.976 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:17.976 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:17.976 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.971+0000 7f75fe1bc640 1 Processor -- start 2026-02-22T23:26:17.976 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 1 -- start start 2026-02-22T23:26:17.976 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient: init 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient: _add_conns ranks=[5,3,1,4,0,2] 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f75f806ff20 0x7f75f8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient(hunting): picked mon.noname-c con 0x7f75f806ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f75f80c99a0 0x7f75f8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient(hunting): picked mon.noname-a con 0x7f75f80c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f75f815a410 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75f815a860 con 0x7f75f815a410 2026-02-22T23:26:17.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f75f8159a40 con 0x7f75f80c99a0 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f75f80c7e90 con 0x7f75f806ff20 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75fe1bc640 10 monclient(hunting): authenticate will time out at 1252.700548s 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.972+0000 7f75f6ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f75f80c99a0 0x7f75f8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f7fff640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f75f815a410 0x7f75f814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:57112/0 (socket says 10.20.193.6:57112) 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f7fff640 1 -- 10.20.193.6:0/1973110956 learned_addr learned my addr 10.20.193.6:0/1973110956 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 1 --2- 10.20.193.6:0/1973110956 >> v2:10.20.193.80:3300/0 conn(0x7f75f806ff20 0x7f75f8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f6ffd640 10 monclient(hunting): get_auth_request con 0x7f75f80c99a0 auth_method 0 2026-02-22T23:26:17.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 10 monclient(hunting): get_auth_request con 0x7f75f806ff20 auth_method 0 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.973+0000 7f75f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f67fc640 1 -- 10.20.193.6:0/1973110956 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3518450145 0 0) 0x7f75f815a860 con 0x7f75f815a410 2026-02-22T23:26:17.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f67fc640 10 monclient(hunting): _negotiate 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f67fc640 10 monclient(hunting): my global_id is 4852 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f67fc640 1 -- 10.20.193.6:0/1973110956 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75dc003620 con 0x7f75f815a410 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_done global_id 4878 payload 274 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 1 -- 10.20.193.6:0/1973110956 >> v2:10.20.193.80:3300/0 conn(0x7f75f806ff20 msgr2=0x7f75f8070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 1 --2- 10.20.193.6:0/1973110956 >> v2:10.20.193.80:3300/0 conn(0x7f75f806ff20 0x7f75f8070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 1 -- 10.20.193.6:0/1973110956 >> v1:10.20.193.40:6789/0 conn(0x7f75f815a410 legacy=0x7f75f814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 1 monclient: found mon.noname-a 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:17.982 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:17.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.974+0000 7f75f6ffd640 1 -- 10.20.193.6:0/1973110956 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f80c8170 con 0x7f75f80c99a0 2026-02-22T23:26:17.983 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.975+0000 7f75f6ffd640 1 --2- 10.20.193.6:0/1973110956 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f8159ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f75e80098a0 tx=0x7f75e802ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684c2de921853394 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:17.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 1 -- 10.20.193.6:0/1973110956 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e803c070 con 0x7f75f80c99a0 2026-02-22T23:26:17.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:17.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:17.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: dump: 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: _finish_auth 0 2026-02-22T23:26:17.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 1 -- 10.20.193.6:0/1973110956 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e802fae0 con 0x7f75f80c99a0 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 1 -- 10.20.193.6:0/1973110956 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8037440 con 0x7f75f80c99a0 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 5 monclient: authenticate success, global_id 4878 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75f67fc640 10 monclient: dump: 2026-02-22T23:26:17.991 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:17.992 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:17.992 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:17.992 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:17.992 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:17.992 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:17.992 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:17.992 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:17.993 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:17.993 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:17.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:17.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 4 set_mon_vals no callback set 2026-02-22T23:26:17.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 10 monclient: shutdown 2026-02-22T23:26:17.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 -- 10.20.193.6:0/1973110956 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 msgr2=0x7f75f8159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/1973110956 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f8159ed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f75e80098a0 tx=0x7f75e802ee10 comp rx=0 tx=0).stop 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 -- 10.20.193.6:0/1973110956 shutdown_connections 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/1973110956 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f8159ed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/1973110956 >> v2:10.20.193.80:3300/0 conn(0x7f75f806ff20 0x7f75f8070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 -- 10.20.193.6:0/1973110956 >> 10.20.193.6:0/1973110956 conn(0x7f75f80cd780 msgr2=0x7f75f80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 -- 10.20.193.6:0/1973110956 shutdown_connections 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.977+0000 7f75fe1bc640 1 -- 10.20.193.6:0/1973110956 wait complete. 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient: build_initial_monmap 2026-02-22T23:26:17.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient: monmap: 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 librados: starting msgr at 2026-02-22T23:26:17.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 librados: starting objecter 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 Processor -- start 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 -- start start 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 librados: setting wanted keys 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 librados: calling monclient init 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient: init 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75f806ff20 0x7f75f80c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): picked mon.noname-c con 0x7f75f806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f80c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): picked mon.noname-a con 0x7f75f80c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75f815a410 0x7f75f814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:17.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): picked mon.noname-b con 0x7f75f815a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f75f80c8a10 con 0x7f75f80c99a0 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f75f80c8730 con 0x7f75f815a410 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f75f8150520 con 0x7f75f806ff20 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.978+0000 7f75fe1bc640 10 monclient(hunting): authenticate will time out at 1252.706728s 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f80c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75f815a410 0x7f75f814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f80c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47702/0 (socket says 10.20.193.6:47702) 2026-02-22T23:26:17.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 1 -- 10.20.193.6:0/169386664 learned_addr learned my addr 10.20.193.6:0/169386664 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 10 monclient(hunting): get_auth_request con 0x7f75f80c99a0 auth_method 0 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75f806ff20 0x7f75f80c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 10 monclient(hunting): get_auth_request con 0x7f75f815a410 auth_method 0 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 10 monclient(hunting): get_auth_request con 0x7f75f806ff20 auth_method 0 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:17.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:17.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.979+0000 7f75f7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 10 monclient(hunting): handle_auth_done global_id 4881 payload 824 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 1 -- 10.20.193.6:0/169386664 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75f806ff20 msgr2=0x7f75f80c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75f806ff20 0x7f75f80c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 1 -- 10.20.193.6:0/169386664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75f815a410 msgr2=0x7f75f814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75f815a410 0x7f75f814e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 1 monclient: found mon.noname-a 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.980+0000 7f75f6ffd640 1 -- 10.20.193.6:0/169386664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f814e9c0 con 0x7f75f80c99a0 2026-02-22T23:26:18.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.981+0000 7f75f6ffd640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f80c1ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75e802f340 tx=0x7f75e80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.981+0000 7f75d3fff640 10 client.?.objecter ms_handle_connect 0x7f75f80c99a0 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.981+0000 7f75d3fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 1 -- 10.20.193.6:0/169386664 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8046020 con 0x7f75f80c99a0 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: dump: 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:18.001 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: _finish_auth 0 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 1 -- 10.20.193.6:0/169386664 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e8042430 con 0x7f75f80c99a0 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 1 -- 10.20.193.6:0/169386664 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e803c040 con 0x7f75f80c99a0 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 5 monclient: authenticate success, global_id 4881 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 10 monclient: _renew_subs 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:18.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f80c2240 con 0x7f75f80c99a0 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75f5ffb640 4 set_mon_vals no callback set 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75d3fff640 10 monclient: dump: 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:18.003 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 10 client.4881.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 10 monclient: _renew_subs 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f80c2800 con 0x7f75f80c99a0 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.983+0000 7f75fe1bc640 1 librados: init done 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.984+0000 7f75fe1bc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.984+0000 7f75fe1bc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.984+0000 7f75fe1bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:18.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.984+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f8070300 con 0x7f75f80c99a0 2026-02-22T23:26:18.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.995+0000 7f75d3fff640 1 -- 10.20.193.6:0/169386664 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f75e8053070 con 0x7f75f80c99a0 2026-02-22T23:26:18.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.995+0000 7f75d3fff640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f75c8078e80 0x7f75c807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:18.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75f77fe640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f75c8078e80 0x7f75c807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75f77fe640 10 monclient: get_auth_request con 0x7f75c8078e80 auth_method 0 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75d3fff640 1 -- 10.20.193.6:0/169386664 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f75e8083460 con 0x7f75f80c99a0 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75d3fff640 10 client.4881.objecter ms_dispatch2 0x7f75f8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75d3fff640 3 client.4881.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75d3fff640 3 client.4881.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75d3fff640 20 client.4881.objecter dump_active .. 0 homeless 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75f77fe640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f75c8078e80 0x7f75c807b340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009c40 tx=0x7f75e4009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:18.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:17.996+0000 7f75d3fff640 10 client.4881.objecter ms_handle_connect 0x7f75c8078e80 2026-02-22T23:26:18.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.091+0000 7f75d3fff640 1 -- 10.20.193.6:0/169386664 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f75e8050020 con 0x7f75f80c99a0 2026-02-22T23:26:18.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.091+0000 7f75d3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:18.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.091+0000 7f75d3fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:18.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.191+0000 7f75fe1bc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:18.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.191+0000 7f75fe1bc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:18.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.191+0000 7f75fe1bc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:18.194 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.191+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f75f80cde00 con 0x7f75f80c99a0 2026-02-22T23:26:18.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.194+0000 7f75d3fff640 1 -- 10.20.193.6:0/169386664 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f75e8087420 con 0x7f75f80c99a0 2026-02-22T23:26:18.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.194+0000 7f75d3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:18.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.194+0000 7f75d3fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:18.197 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:18.197 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:18.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 10 librados: watch_flush enter 2026-02-22T23:26:18.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 10 librados: watch_flush exit 2026-02-22T23:26:18.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 20 client.4881.objecter shutdown clearing up homeless session... 2026-02-22T23:26:18.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 10 client.4881.objecter successfully canceled tick 2026-02-22T23:26:18.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f75c8078e80 msgr2=0x7f75c807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:18.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f75c8078e80 0x7f75c807b340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009c40 tx=0x7f75e4009290 comp rx=0 tx=0).stop 2026-02-22T23:26:18.199 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 10 monclient: shutdown 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 msgr2=0x7f75f80c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f80c1ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75e802f340 tx=0x7f75e80047c0 comp rx=0 tx=0).stop 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 shutdown_connections 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f75c8078e80 0x7f75c807b340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f75f815a410 0x7f75f814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f75f80c99a0 0x7f75f80c1ae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 --2- 10.20.193.6:0/169386664 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f75f806ff20 0x7f75f80c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 >> 10.20.193.6:0/169386664 conn(0x7f75f80cd780 msgr2=0x7f75f80c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:18.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 shutdown_connections 2026-02-22T23:26:18.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 -- 10.20.193.6:0/169386664 wait complete. 2026-02-22T23:26:18.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:18.195+0000 7f75fe1bc640 1 librados: shutdown 2026-02-22T23:26:21.208 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:21.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.273+0000 7f3b3a2ea640 10 monclient: get_monmap_and_config 2026-02-22T23:26:21.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.273+0000 7f3b3a2ea640 10 monclient: build_initial_monmap 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.273+0000 7f3b3a2ea640 10 monclient: monmap: 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:21.275230+0000 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:21.275230+0000 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:21.277 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:21.278 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:21.278 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:21.278 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:21.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 1 Processor -- start 2026-02-22T23:26:21.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 1 -- start start 2026-02-22T23:26:21.278 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient: init 2026-02-22T23:26:21.280 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient: _add_conns ranks=[4,0,3,1,5,2] 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f3b3406ff20 0x7f3b34070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient(hunting): picked mon.noname-b con 0x7f3b3406ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3b340c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f3b3415a5e0 0x7f3b3414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient(hunting): picked mon.noname-a con 0x7f3b3415a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b340c7e90 con 0x7f3b340c99a0 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f3b34159a40 con 0x7f3b3415a5e0 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:21.281 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f3b340c8170 con 0x7f3b3406ff20 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.274+0000 7f3b3a2ea640 10 monclient(hunting): authenticate will time out at 1256.002729s 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f3b3406ff20 0x7f3b34070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b38860640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f3b3415a5e0 0x7f3b3414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b337fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f3b340c99a0 0x7f3b34159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52538/0 (socket says 10.20.193.6:52538) 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b337fe640 1 -- 10.20.193.6:0/3722952635 learned_addr learned my addr 10.20.193.6:0/3722952635 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 10 monclient(hunting): get_auth_request con 0x7f3b3406ff20 auth_method 0 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:21.282 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b38860640 10 monclient(hunting): get_auth_request con 0x7f3b3415a5e0 auth_method 0 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b38860640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b38860640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b38860640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b38860640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.275+0000 7f3b38860640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b38860640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 10 monclient(hunting): handle_auth_done global_id 4858 payload 274 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 1 -- 10.20.193.6:0/3722952635 >> v2:10.20.193.6:3300/0 conn(0x7f3b3415a5e0 msgr2=0x7f3b3414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 1 --2- 10.20.193.6:0/3722952635 >> v2:10.20.193.6:3300/0 conn(0x7f3b3415a5e0 0x7f3b3414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 1 -- 10.20.193.6:0/3722952635 >> v1:10.20.193.6:6789/0 conn(0x7f3b340c99a0 legacy=0x7f3b34159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:21.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:21.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 1 monclient: found mon.noname-b 2026-02-22T23:26:21.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:21.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:21.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.276+0000 7f3b33fff640 1 -- 10.20.193.6:0/3722952635 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b340c8450 con 0x7f3b3406ff20 2026-02-22T23:26:21.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.277+0000 7f3b33fff640 1 --2- 10.20.193.6:0/3722952635 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b3406ff20 0x7f3b34070300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b200098a0 tx=0x7f3b2002ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea59e4fc3e2ce75d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:21.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 1 -- 10.20.193.6:0/3722952635 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2003c070 con 0x7f3b3406ff20 2026-02-22T23:26:21.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: dump: 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:21.291 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: _finish_auth 0 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 1 -- 10.20.193.6:0/3722952635 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2002fae0 con 0x7f3b3406ff20 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b3a2ea640 5 monclient: authenticate success, global_id 4858 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b3a2ea640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 1 -- 10.20.193.6:0/3722952635 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20037440 con 0x7f3b3406ff20 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b3a2ea640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b3a2ea640 4 set_mon_vals no callback set 2026-02-22T23:26:21.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.279+0000 7f3b32ffd640 10 monclient: dump: 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:21.293 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 10 monclient: shutdown 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/3722952635 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b3406ff20 msgr2=0x7f3b34070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/3722952635 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b3406ff20 0x7f3b34070300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b200098a0 tx=0x7f3b2002ee10 comp rx=0 tx=0).stop 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/3722952635 shutdown_connections 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/3722952635 >> v2:10.20.193.6:3300/0 conn(0x7f3b3415a5e0 0x7f3b3414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/3722952635 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b3406ff20 0x7f3b34070300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/3722952635 >> 10.20.193.6:0/3722952635 conn(0x7f3b340cd780 msgr2=0x7f3b340cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/3722952635 shutdown_connections 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/3722952635 wait complete. 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 10 monclient: build_initial_monmap 2026-02-22T23:26:21.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.280+0000 7f3b3a2ea640 10 monclient: monmap: 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 librados: starting msgr at 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 librados: starting objecter 2026-02-22T23:26:21.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 Processor -- start 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 -- start start 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 librados: setting wanted keys 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 librados: calling monclient init 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient: init 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3b3406ff20 0x7f3b341f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): picked mon.noname-a con 0x7f3b3406ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b340c99a0 0x7f3b3406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): picked mon.noname-b con 0x7f3b340c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3b3415a5e0 0x7f3b341f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): picked mon.noname-c con 0x7f3b3415a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:21.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3b34150350 con 0x7f3b3406ff20 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3b340c8a10 con 0x7f3b340c99a0 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3b341504d0 con 0x7f3b3415a5e0 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b3a2ea640 10 monclient(hunting): authenticate will time out at 1256.009672s 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b33fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3b3406ff20 0x7f3b341f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b33fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3b3406ff20 0x7f3b341f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47734/0 (socket says 10.20.193.6:47734) 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b33fff640 1 -- 10.20.193.6:0/2140580953 learned_addr learned my addr 10.20.193.6:0/2140580953 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b33fff640 10 monclient(hunting): get_auth_request con 0x7f3b3406ff20 auth_method 0 2026-02-22T23:26:21.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b33fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b33fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.281+0000 7f3b33fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b340c99a0 0x7f3b3406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b33fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b33fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b33fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3b3415a5e0 0x7f3b341f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 10 monclient(hunting): get_auth_request con 0x7f3b340c99a0 auth_method 0 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:21.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b337fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 10 monclient(hunting): get_auth_request con 0x7f3b3415a5e0 auth_method 0 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.282+0000 7f3b38860640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 10 monclient(hunting): handle_auth_done global_id 4890 payload 824 2026-02-22T23:26:21.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 1 -- 10.20.193.6:0/2140580953 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3b3415a5e0 msgr2=0x7f3b341f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3b3415a5e0 0x7f3b341f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 1 -- 10.20.193.6:0/2140580953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b340c99a0 msgr2=0x7f3b3406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b340c99a0 0x7f3b3406eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 1 monclient: found mon.noname-a 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.283+0000 7f3b33fff640 1 -- 10.20.193.6:0/2140580953 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3406e130 con 0x7f3b3406ff20 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.284+0000 7f3b33fff640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3b3406ff20 0x7f3b341f4ba0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b2002f340 tx=0x7f3b200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.284+0000 7f3b30ff9640 10 client.?.objecter ms_handle_connect 0x7f3b3406ff20 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.284+0000 7f3b30ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:21.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 1 -- 10.20.193.6:0/2140580953 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2003c040 con 0x7f3b3406ff20 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: dump: 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:21.301 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: _finish_auth 0 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 1 -- 10.20.193.6:0/2140580953 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b20042450 con 0x7f3b3406ff20 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 5 monclient: authenticate success, global_id 4890 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 10 monclient: _renew_subs 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b341f3fb0 con 0x7f3b3406ff20 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 1 -- 10.20.193.6:0/2140580953 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20042840 con 0x7f3b3406ff20 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:21.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b327fc640 4 set_mon_vals no callback set 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b30ff9640 10 monclient: dump: 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:21.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 10 client.4890.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 10 monclient: _renew_subs 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b341f4570 con 0x7f3b3406ff20 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.286+0000 7f3b3a2ea640 1 librados: init done 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.287+0000 7f3b3a2ea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.287+0000 7f3b3a2ea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.287+0000 7f3b3a2ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:21.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.287+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b34150860 con 0x7f3b3406ff20 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.298+0000 7f3b30ff9640 1 -- 10.20.193.6:0/2140580953 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3b200429e0 con 0x7f3b3406ff20 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.298+0000 7f3b30ff9640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3b04078e80 0x7f3b0407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.298+0000 7f3b337fe640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3b04078e80 0x7f3b0407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.298+0000 7f3b337fe640 10 monclient: get_auth_request con 0x7f3b04078e80 auth_method 0 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.299+0000 7f3b337fe640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3b04078e80 0x7f3b0407b340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3b34158b40 tx=0x7f3b24009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.299+0000 7f3b30ff9640 10 client.4890.objecter ms_handle_connect 0x7f3b04078e80 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.300+0000 7f3b30ff9640 1 -- 10.20.193.6:0/2140580953 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f3b2002f9a0 con 0x7f3b3406ff20 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.300+0000 7f3b30ff9640 10 client.4890.objecter ms_dispatch2 0x7f3b34000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.300+0000 7f3b30ff9640 3 client.4890.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:21.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.300+0000 7f3b30ff9640 3 client.4890.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:21.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.300+0000 7f3b30ff9640 20 client.4890.objecter dump_active .. 0 homeless 2026-02-22T23:26:21.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.396+0000 7f3b30ff9640 1 -- 10.20.193.6:0/2140580953 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3b20087530 con 0x7f3b3406ff20 2026-02-22T23:26:21.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.396+0000 7f3b30ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:21.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.396+0000 7f3b30ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:21.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.496+0000 7f3b3a2ea640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:21.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.496+0000 7f3b3a2ea640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:21.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.496+0000 7f3b3a2ea640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:21.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.496+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3b3414f630 con 0x7f3b3406ff20 2026-02-22T23:26:21.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.499+0000 7f3b30ff9640 1 -- 10.20.193.6:0/2140580953 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3b2008b830 con 0x7f3b3406ff20 2026-02-22T23:26:21.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.499+0000 7f3b30ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:21.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.499+0000 7f3b30ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:21.503 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:21.503 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:21.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 10 librados: watch_flush enter 2026-02-22T23:26:21.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 10 librados: watch_flush exit 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 20 client.4890.objecter shutdown clearing up homeless session... 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 10 client.4890.objecter successfully canceled tick 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3b04078e80 msgr2=0x7f3b0407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3b04078e80 0x7f3b0407b340 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3b34158b40 tx=0x7f3b24009040 comp rx=0 tx=0).stop 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 10 monclient: shutdown 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3b3406ff20 msgr2=0x7f3b341f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3b3406ff20 0x7f3b341f4ba0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3b2002f340 tx=0x7f3b200047c0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 shutdown_connections 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3b04078e80 0x7f3b0407b340 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3b3415a5e0 0x7f3b341f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3b340c99a0 0x7f3b3406eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 --2- 10.20.193.6:0/2140580953 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3b3406ff20 0x7f3b341f4ba0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:21.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 >> 10.20.193.6:0/2140580953 conn(0x7f3b340cd780 msgr2=0x7f3b3414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:21.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 shutdown_connections 2026-02-22T23:26:21.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 -- 10.20.193.6:0/2140580953 wait complete. 2026-02-22T23:26:21.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:21.501+0000 7f3b3a2ea640 1 librados: shutdown 2026-02-22T23:26:24.513 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:24.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.578+0000 7f1109ff7640 10 monclient: get_monmap_and_config 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.578+0000 7f1109ff7640 10 monclient: build_initial_monmap 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.578+0000 7f1109ff7640 10 monclient: monmap: 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:24.580231+0000 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:24.580231+0000 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:24.581 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:24.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 1 Processor -- start 2026-02-22T23:26:24.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 1 -- start start 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient: init 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient: _add_conns ranks=[2,3,1,0,5,4] 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f110415b860 addr v1:10.20.193.80:6789/0 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f11040740a0 0x7f110415bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient(hunting): picked mon.noname-a con 0x7f11040740a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1104072480 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1104072d20 con 0x7f1104072480 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1104073000 con 0x7f110415b860 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f11040732e0 con 0x7f11040740a0 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.579+0000 7f1109ff7640 10 monclient(hunting): authenticate will time out at 1259.307700s 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f11040740a0 0x7f110415bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f11037fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f110415b860 0x7f11041594f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:41070/0 (socket says 10.20.193.6:41070) 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f11037fe640 1 -- 10.20.193.6:0/3483497571 learned_addr learned my addr 10.20.193.6:0/3483497571 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 10 monclient(hunting): get_auth_request con 0x7f11040740a0 auth_method 0 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:24.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.580+0000 7f1102ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 10 monclient(hunting): handle_auth_done global_id 4893 payload 274 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 1 -- 10.20.193.6:0/3483497571 >> v1:10.20.193.80:6789/0 conn(0x7f110415b860 legacy=0x7f11041594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 1 -- 10.20.193.6:0/3483497571 >> v1:10.20.193.40:6789/0 conn(0x7f1104072480 legacy=0x7f11040c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 1 monclient: found mon.noname-a 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.581+0000 7f1102ffd640 1 -- 10.20.193.6:0/3483497571 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11040735c0 con 0x7f11040740a0 2026-02-22T23:26:24.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.582+0000 7f1102ffd640 1 --2- 10.20.193.6:0/3483497571 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11040740a0 0x7f110415bc40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f10f400aab0 tx=0x7f10f4031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b3902a56adcefb server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:24.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 1 -- 10.20.193.6:0/3483497571 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4036710 con 0x7f11040740a0 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: dump: 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: _finish_auth 0 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: _check_auth_tickets 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 5 monclient: authenticate success, global_id 4893 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 1 -- 10.20.193.6:0/3483497571 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f40368b0 con 0x7f11040740a0 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:24.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 1 -- 10.20.193.6:0/3483497571 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4036c80 con 0x7f11040740a0 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 4 set_mon_vals no callback set 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f11027fc640 10 monclient: dump: 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 10 monclient: shutdown 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 1 -- 10.20.193.6:0/3483497571 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11040740a0 msgr2=0x7f110415bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 1 --2- 10.20.193.6:0/3483497571 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11040740a0 0x7f110415bc40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f10f400aab0 tx=0x7f10f4031450 comp rx=0 tx=0).stop 2026-02-22T23:26:24.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 1 -- 10.20.193.6:0/3483497571 shutdown_connections 2026-02-22T23:26:24.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 1 --2- 10.20.193.6:0/3483497571 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f11040740a0 0x7f110415bc40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:24.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 1 -- 10.20.193.6:0/3483497571 >> 10.20.193.6:0/3483497571 conn(0x7f11040cd780 msgr2=0x7f11040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:24.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 1 -- 10.20.193.6:0/3483497571 shutdown_connections 2026-02-22T23:26:24.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.584+0000 7f1109ff7640 1 -- 10.20.193.6:0/3483497571 wait complete. 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient: build_initial_monmap 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient: monmap: 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:24.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 librados: starting msgr at 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 librados: starting objecter 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 Processor -- start 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 -- start start 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 librados: setting wanted keys 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 librados: calling monclient init 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient: init 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1104072480 0x7f1104155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): picked mon.noname-c con 0x7f1104072480 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11040740a0 0x7f11041f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): picked mon.noname-b con 0x7f11040740a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f110415b860 0x7f11041f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): picked mon.noname-a con 0x7f110415b860 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f11040728f0 con 0x7f110415b860 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1104073b80 con 0x7f11040740a0 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:24.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1104072a70 con 0x7f1104072480 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.585+0000 7f1109ff7640 10 monclient(hunting): authenticate will time out at 1259.313958s 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f110415b860 0x7f11041f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11040740a0 0x7f11041f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f110415b860 0x7f11041f2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47756/0 (socket says 10.20.193.6:47756) 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 1 -- 10.20.193.6:0/979742303 learned_addr learned my addr 10.20.193.6:0/979742303 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 10 monclient(hunting): get_auth_request con 0x7f110415b860 auth_method 0 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1104072480 0x7f1104155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 10 monclient(hunting): get_auth_request con 0x7f11040740a0 auth_method 0 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1103fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 10 monclient(hunting): get_auth_request con 0x7f1104072480 auth_method 0 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:24.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f1102ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.586+0000 7f11037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 10 monclient(hunting): handle_auth_done global_id 4896 payload 824 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 1 -- 10.20.193.6:0/979742303 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1104072480 msgr2=0x7f1104155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1104072480 0x7f1104155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 1 -- 10.20.193.6:0/979742303 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11040740a0 msgr2=0x7f11041f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11040740a0 0x7f11041f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 1 monclient: found mon.noname-a 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.588+0000 7f1103fff640 1 -- 10.20.193.6:0/979742303 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1104155c50 con 0x7f110415b860 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f1103fff640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f110415b860 0x7f11041f2900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10f800cbd0 tx=0x7f10f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 10 client.?.objecter ms_handle_connect 0x7f110415b860 2026-02-22T23:26:24.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 1 -- 10.20.193.6:0/979742303 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f800ad50 con 0x7f110415b860 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 10 monclient: dump: 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 10 monclient: _finish_auth 0 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 10 monclient: _check_auth_tickets 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f10dffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f1109ff7640 5 monclient: authenticate success, global_id 4896 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f1109ff7640 10 monclient: _renew_subs 2026-02-22T23:26:24.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f1109ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.589+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11041f6570 con 0x7f110415b860 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 10 client.4896.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 10 monclient: _renew_subs 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11041f18e0 con 0x7f110415b860 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 1 librados: init done 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110415bcc0 con 0x7f110415b860 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f10dffff640 1 -- 10.20.193.6:0/979742303 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f8003a70 con 0x7f110415b860 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f10dffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f10dffff640 1 -- 10.20.193.6:0/979742303 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8003e20 con 0x7f110415b860 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f10dffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f10dffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f10dffff640 10 monclient: dump: 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:24.597 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:24.598 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:24.598 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:24.598 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:24.598 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:24.598 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:24.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.590+0000 7f1101ffb640 4 set_mon_vals no callback set 2026-02-22T23:26:24.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.695+0000 7f10dffff640 1 -- 10.20.193.6:0/979742303 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f10f80050d0 con 0x7f110415b860 2026-02-22T23:26:24.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.696+0000 7f10dffff640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f10d4078e00 0x7f10d407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:24.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.696+0000 7f11037fe640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f10d4078e00 0x7f10d407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:24.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.696+0000 7f11037fe640 10 monclient: get_auth_request con 0x7f10d4078e00 auth_method 0 2026-02-22T23:26:24.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.697+0000 7f10dffff640 10 client.4896.objecter ms_handle_connect 0x7f10d4078e00 2026-02-22T23:26:24.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.697+0000 7f11037fe640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f10d4078e00 0x7f10d407b2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10f0002770 tx=0x7f10f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 1 -- 10.20.193.6:0/979742303 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f10f805e700 con 0x7f110415b860 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 10 client.4896.objecter ms_dispatch2 0x7f1104000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 3 client.4896.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 3 client.4896.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 20 client.4896.objecter dump_active .. 0 homeless 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 1 -- 10.20.193.6:0/979742303 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f10f80626c0 con 0x7f110415b860 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:24.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.704+0000 7f10dffff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:24.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.804+0000 7f1109ff7640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:24.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.804+0000 7f1109ff7640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:24.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.804+0000 7f1109ff7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:24.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.804+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f110406f2e0 con 0x7f110415b860 2026-02-22T23:26:24.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.807+0000 7f10dffff640 1 -- 10.20.193.6:0/979742303 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f10f80669c0 con 0x7f110415b860 2026-02-22T23:26:24.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.807+0000 7f10dffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:24.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.807+0000 7f10dffff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:24.809 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:24.809 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:24.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 10 librados: watch_flush enter 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 10 librados: watch_flush exit 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 20 client.4896.objecter shutdown clearing up homeless session... 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 10 client.4896.objecter successfully canceled tick 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f10d4078e00 msgr2=0x7f10d407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f10d4078e00 0x7f10d407b2c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10f0002770 tx=0x7f10f0009040 comp rx=0 tx=0).stop 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 10 monclient: shutdown 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f110415b860 msgr2=0x7f11041f2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f110415b860 0x7f11041f2900 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10f800cbd0 tx=0x7f10f800a430 comp rx=0 tx=0).stop 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 shutdown_connections 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f10d4078e00 0x7f10d407b2c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f110415b860 0x7f11041f2900 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f11040740a0 0x7f11041f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 --2- 10.20.193.6:0/979742303 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1104072480 0x7f1104155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 >> 10.20.193.6:0/979742303 conn(0x7f11040cd780 msgr2=0x7f11040cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 shutdown_connections 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 -- 10.20.193.6:0/979742303 wait complete. 2026-02-22T23:26:24.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:24.808+0000 7f1109ff7640 1 librados: shutdown 2026-02-22T23:26:27.820 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.886+0000 7fa8f6d49640 10 monclient: get_monmap_and_config 2026-02-22T23:26:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.886+0000 7fa8f6d49640 10 monclient: build_initial_monmap 2026-02-22T23:26:27.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.886+0000 7fa8f6d49640 10 monclient: monmap: 2026-02-22T23:26:27.888 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:27.888 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:27.887021+0000 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:27.887021+0000 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:27.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.886+0000 7fa8f6d49640 1 Processor -- start 2026-02-22T23:26:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.886+0000 7fa8f6d49640 1 -- start start 2026-02-22T23:26:27.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.886+0000 7fa8f6d49640 10 monclient: init 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient: _add_conns ranks=[4,5,3,2,1,0] 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa8f006ff20 0x7fa8f0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): picked mon.noname-b con 0x7fa8f006ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa8f00c99a0 0x7fa8f0159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): picked mon.noname-c con 0x7fa8f00c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa8f015a410 0x7fa8f014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:27.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): picked mon.noname-a con 0x7fa8f015a410 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f0159a40 con 0x7fa8f015a410 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f015a860 con 0x7fa8f006ff20 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f00c7e90 con 0x7fa8f00c99a0 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f6d49640 10 monclient(hunting): authenticate will time out at 1262.614384s 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f52bf640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa8f015a410 0x7fa8f014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f52bf640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa8f015a410 0x7fa8f014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47762/0 (socket says 10.20.193.6:47762) 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f52bf640 1 -- 10.20.193.6:0/3603955591 learned_addr learned my addr 10.20.193.6:0/3603955591 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:27.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f52bf640 10 monclient(hunting): get_auth_request con 0x7fa8f015a410 auth_method 0 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f52bf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f52bf640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.887+0000 7fa8f52bf640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.888+0000 7fa8f52bf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.888+0000 7fa8f52bf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.888+0000 7fa8f52bf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 10 monclient(hunting): handle_auth_done global_id 4899 payload 274 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 1 -- 10.20.193.6:0/3603955591 >> v2:10.20.193.80:3300/0 conn(0x7fa8f00c99a0 msgr2=0x7fa8f0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 1 --2- 10.20.193.6:0/3603955591 >> v2:10.20.193.80:3300/0 conn(0x7fa8f00c99a0 0x7fa8f0159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 1 -- 10.20.193.6:0/3603955591 >> v2:10.20.193.40:3300/0 conn(0x7fa8f006ff20 msgr2=0x7fa8f0070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 1 --2- 10.20.193.6:0/3603955591 >> v2:10.20.193.40:3300/0 conn(0x7fa8f006ff20 0x7fa8f0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 10 monclient: _finish_hunting 0 2026-02-22T23:26:27.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 1 monclient: found mon.noname-a 2026-02-22T23:26:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 1 -- 10.20.193.6:0/3603955591 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f00c8170 con 0x7fa8f015a410 2026-02-22T23:26:27.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.889+0000 7fa8f52bf640 1 --2- 10.20.193.6:0/3603955591 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f015a410 0x7fa8f014fe10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00709c0 tx=0x7fa8ec02f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f3fb63de2ea347 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 1 -- 10.20.193.6:0/3603955591 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec02fce0 con 0x7fa8f015a410 2026-02-22T23:26:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:27.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: dump: 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: _finish_auth 0 2026-02-22T23:26:27.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 1 -- 10.20.193.6:0/3603955591 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ec0373f0 con 0x7fa8f015a410 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8f6d49640 5 monclient: authenticate success, global_id 4899 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8f6d49640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 1 -- 10.20.193.6:0/3603955591 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0377c0 con 0x7fa8f015a410 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8f6d49640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8f6d49640 4 set_mon_vals no callback set 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.891+0000 7fa8e77fe640 10 monclient: dump: 2026-02-22T23:26:27.902 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 10 monclient: shutdown 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:27.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 -- 10.20.193.6:0/3603955591 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f015a410 msgr2=0x7fa8f014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/3603955591 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f015a410 0x7fa8f014fe10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa8f00709c0 tx=0x7fa8ec02f270 comp rx=0 tx=0).stop 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 -- 10.20.193.6:0/3603955591 shutdown_connections 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/3603955591 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f015a410 0x7fa8f014fe10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/3603955591 >> v2:10.20.193.80:3300/0 conn(0x7fa8f00c99a0 0x7fa8f0159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/3603955591 >> v2:10.20.193.40:3300/0 conn(0x7fa8f006ff20 0x7fa8f0070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 -- 10.20.193.6:0/3603955591 >> 10.20.193.6:0/3603955591 conn(0x7fa8f00cd780 msgr2=0x7fa8f00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 -- 10.20.193.6:0/3603955591 shutdown_connections 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 -- 10.20.193.6:0/3603955591 wait complete. 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 10 monclient: build_initial_monmap 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 10 monclient: monmap: 2026-02-22T23:26:27.904 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 librados: starting msgr at 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.892+0000 7fa8f6d49640 1 librados: starting objecter 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 Processor -- start 2026-02-22T23:26:27.905 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 -- start start 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 librados: setting wanted keys 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 librados: calling monclient init 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient: init 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8f006ff20 0x7fa8f01f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): picked mon.noname-c con 0x7fa8f006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f00c99a0 0x7fa8f014bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): picked mon.noname-a con 0x7fa8f00c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8f015a410 0x7fa8f00c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): picked mon.noname-b con 0x7fa8f015a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:27.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f00c8a10 con 0x7fa8f00c99a0 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f00c8730 con 0x7fa8f015a410 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0150350 con 0x7fa8f006ff20 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8f6d49640 10 monclient(hunting): authenticate will time out at 1262.620615s 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8e7fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f00c99a0 0x7fa8f014bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8e7fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f00c99a0 0x7fa8f014bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:47778/0 (socket says 10.20.193.6:47778) 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8e7fff640 1 -- 10.20.193.6:0/386689991 learned_addr learned my addr 10.20.193.6:0/386689991 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8e7fff640 10 monclient(hunting): get_auth_request con 0x7fa8f00c99a0 auth_method 0 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:27.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.893+0000 7fa8e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.894+0000 7fa8e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.894+0000 7fa8e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.894+0000 7fa8e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 10 monclient(hunting): handle_auth_done global_id 4902 payload 824 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 1 -- 10.20.193.6:0/386689991 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8f006ff20 msgr2=0x7fa8f01f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8f006ff20 0x7fa8f01f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 1 -- 10.20.193.6:0/386689991 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8f015a410 msgr2=0x7fa8f00c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8f015a410 0x7fa8f00c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 1 monclient: found mon.noname-a 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:27.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.895+0000 7fa8e7fff640 1 -- 10.20.193.6:0/386689991 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f00c2500 con 0x7fa8f00c99a0 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.896+0000 7fa8e7fff640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f00c99a0 0x7fa8f014bb30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800b780 tx=0x7fa8d800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.896+0000 7fa8e57fa640 10 client.?.objecter ms_handle_connect 0x7fa8f00c99a0 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.896+0000 7fa8e57fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 1 -- 10.20.193.6:0/386689991 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8004480 con 0x7fa8f00c99a0 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: dump: 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.909 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: _finish_auth 0 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: _check_auth_tickets 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 1 -- 10.20.193.6:0/386689991 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d8010070 con 0x7fa8f00c99a0 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 1 -- 10.20.193.6:0/386689991 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8002920 con 0x7fa8f00c99a0 2026-02-22T23:26:27.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e6ffd640 4 set_mon_vals no callback set 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8e57fa640 10 monclient: dump: 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:27.911 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 5 monclient: authenticate success, global_id 4902 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 monclient: _renew_subs 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f00c1f70 con 0x7fa8f00c99a0 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 client.4902.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 monclient: _renew_subs 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f00c2aa0 con 0x7fa8f00c99a0 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 1 librados: init done 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:27.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.898+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f0150700 con 0x7fa8f00c99a0 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.909+0000 7fa8e57fa640 1 -- 10.20.193.6:0/386689991 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa8d801d030 con 0x7fa8f00c99a0 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.909+0000 7fa8e57fa640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa8c0078e50 0x7fa8c007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.910+0000 7fa8f4abe640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa8c0078e50 0x7fa8c007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.910+0000 7fa8f4abe640 10 monclient: get_auth_request con 0x7fa8c0078e50 auth_method 0 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.910+0000 7fa8e57fa640 10 client.4902.objecter ms_handle_connect 0x7fa8c0078e50 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.910+0000 7fa8f4abe640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa8c0078e50 0x7fa8c007b310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00098a0 tx=0x7fa8e0006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.911+0000 7fa8e57fa640 1 -- 10.20.193.6:0/386689991 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fa8d801de30 con 0x7fa8f00c99a0 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.911+0000 7fa8e57fa640 10 client.4902.objecter ms_dispatch2 0x7fa8f0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.911+0000 7fa8e57fa640 3 client.4902.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:27.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.911+0000 7fa8e57fa640 3 client.4902.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:27.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:27.911+0000 7fa8e57fa640 20 client.4902.objecter dump_active .. 0 homeless 2026-02-22T23:26:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.009+0000 7fa8e57fa640 1 -- 10.20.193.6:0/386689991 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa8d8024080 con 0x7fa8f00c99a0 2026-02-22T23:26:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.009+0000 7fa8e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:28.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.009+0000 7fa8e57fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:28.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.110+0000 7fa8f6d49640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:28.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.110+0000 7fa8f6d49640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:28.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.110+0000 7fa8f6d49640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:28.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.110+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa8f015a1f0 con 0x7fa8f00c99a0 2026-02-22T23:26:28.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.113+0000 7fa8e57fa640 1 -- 10.20.193.6:0/386689991 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fa8d8061a60 con 0x7fa8f00c99a0 2026-02-22T23:26:28.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.113+0000 7fa8e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:28.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.113+0000 7fa8e57fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:28.114 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:28.115 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:28.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 10 librados: watch_flush enter 2026-02-22T23:26:28.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 10 librados: watch_flush exit 2026-02-22T23:26:28.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 20 client.4902.objecter shutdown clearing up homeless session... 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 10 client.4902.objecter successfully canceled tick 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa8c0078e50 msgr2=0x7fa8c007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa8c0078e50 0x7fa8c007b310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00098a0 tx=0x7fa8e0006d90 comp rx=0 tx=0).stop 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 10 monclient: shutdown 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f00c99a0 msgr2=0x7fa8f014bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f00c99a0 0x7fa8f014bb30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800b780 tx=0x7fa8d800bc50 comp rx=0 tx=0).stop 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 shutdown_connections 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa8c0078e50 0x7fa8c007b310 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8f015a410 0x7fa8f00c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8f00c99a0 0x7fa8f014bb30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:28.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 --2- 10.20.193.6:0/386689991 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8f006ff20 0x7fa8f01f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:28.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 >> 10.20.193.6:0/386689991 conn(0x7fa8f00cd780 msgr2=0x7fa8f0073d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:28.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 shutdown_connections 2026-02-22T23:26:28.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 -- 10.20.193.6:0/386689991 wait complete. 2026-02-22T23:26:28.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:28.114+0000 7fa8f6d49640 1 librados: shutdown 2026-02-22T23:26:31.125 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:31.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.193+0000 7fd857e1a640 10 monclient: get_monmap_and_config 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.193+0000 7fd857e1a640 10 monclient: build_initial_monmap 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.193+0000 7fd857e1a640 10 monclient: monmap: 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:31.194465+0000 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:31.194465+0000 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:31.196 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:31.197 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:31.197 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:31.197 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 1 Processor -- start 2026-02-22T23:26:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 1 -- start start 2026-02-22T23:26:31.197 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient: init 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient: _add_conns ranks=[3,4,2,5,1,0] 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd85006ff20 0x7fd850070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient(hunting): picked mon.noname-a con 0x7fd85006ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fd8500c99a0 0x7fd850159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient(hunting): picked mon.noname-b con 0x7fd8500c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd85015a410 addr v1:10.20.193.80:6789/0 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd85015a860 con 0x7fd85015a410 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:31.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd850159a40 con 0x7fd85006ff20 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd8500c7e90 con 0x7fd8500c99a0 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.194+0000 7fd857e1a640 10 monclient(hunting): authenticate will time out at 1265.921817s 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd85006ff20 0x7fd850070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd85006ff20 0x7fd850070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:44686/0 (socket says 10.20.193.6:44686) 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 1 -- 10.20.193.6:0/2032149100 learned_addr learned my addr 10.20.193.6:0/2032149100 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 10 monclient(hunting): get_auth_request con 0x7fd85006ff20 auth_method 0 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:31.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd85538e640 1 --2- 10.20.193.6:0/2032149100 >> v2:10.20.193.40:3300/0 conn(0x7fd8500c99a0 0x7fd850159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd85538e640 10 monclient(hunting): get_auth_request con 0x7fd8500c99a0 auth_method 0 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd85538e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd85538e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd85538e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.195+0000 7fd855b8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd85538e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd85538e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd85538e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:31.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 10 monclient(hunting): handle_auth_done global_id 4905 payload 274 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 1 -- 10.20.193.6:0/2032149100 >> v2:10.20.193.40:3300/0 conn(0x7fd8500c99a0 msgr2=0x7fd850159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 1 --2- 10.20.193.6:0/2032149100 >> v2:10.20.193.40:3300/0 conn(0x7fd8500c99a0 0x7fd850159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 1 -- 10.20.193.6:0/2032149100 >> v1:10.20.193.80:6789/0 conn(0x7fd85015a410 legacy=0x7fd85014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 10 monclient: _finish_hunting 0 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 1 monclient: found mon.noname-a 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.196+0000 7fd855b8f640 1 -- 10.20.193.6:0/2032149100 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8500c8170 con 0x7fd85006ff20 2026-02-22T23:26:31.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.197+0000 7fd855b8f640 1 --2- 10.20.193.6:0/2032149100 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd85006ff20 0x7fd850070300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd840009970 tx=0x7fd84002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f713105c2ce5dfd0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 1 -- 10.20.193.6:0/2032149100 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84003c070 con 0x7fd85006ff20 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: dump: 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.210 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: _finish_auth 0 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: _check_auth_tickets 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 1 -- 10.20.193.6:0/2032149100 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84002fbc0 con 0x7fd85006ff20 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 5 monclient: authenticate success, global_id 4905 2026-02-22T23:26:31.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 1 -- 10.20.193.6:0/2032149100 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840037440 con 0x7fd85006ff20 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd854b8d640 10 monclient: dump: 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:31.212 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 4 set_mon_vals no callback set 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 10 monclient: shutdown 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 1 -- 10.20.193.6:0/2032149100 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd85006ff20 msgr2=0x7fd850070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 1 --2- 10.20.193.6:0/2032149100 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd85006ff20 0x7fd850070300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd840009970 tx=0x7fd84002eef0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 1 -- 10.20.193.6:0/2032149100 shutdown_connections 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 1 --2- 10.20.193.6:0/2032149100 >> v2:10.20.193.40:3300/0 conn(0x7fd8500c99a0 0x7fd850159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 1 --2- 10.20.193.6:0/2032149100 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd85006ff20 0x7fd850070300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.213 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.199+0000 7fd857e1a640 1 -- 10.20.193.6:0/2032149100 >> 10.20.193.6:0/2032149100 conn(0x7fd8500cd780 msgr2=0x7fd8500cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 -- 10.20.193.6:0/2032149100 shutdown_connections 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 -- 10.20.193.6:0/2032149100 wait complete. 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 10 monclient: build_initial_monmap 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 10 monclient: monmap: 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:31.214 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 librados: starting msgr at 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 librados: starting objecter 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 Processor -- start 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 -- start start 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 librados: setting wanted keys 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 1 librados: calling monclient init 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.200+0000 7fd857e1a640 10 monclient: init 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd85006ff20 0x7fd85014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): picked mon.noname-c con 0x7fd85006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:31.215 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd8500c99a0 0x7fd8501f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): picked mon.noname-a con 0x7fd8500c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd85015a410 0x7fd8501f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): picked mon.noname-b con 0x7fd85015a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd8500c8a10 con 0x7fd8500c99a0 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd8500c8730 con 0x7fd85015a410 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd850150520 con 0x7fd85006ff20 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd857e1a640 10 monclient(hunting): authenticate will time out at 1265.928308s 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd8500c99a0 0x7fd8501f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:31.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd8500c99a0 0x7fd8501f4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:44702/0 (socket says 10.20.193.6:44702) 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 1 -- 10.20.193.6:0/401645665 learned_addr learned my addr 10.20.193.6:0/401645665 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 10 monclient(hunting): get_auth_request con 0x7fd8500c99a0 auth_method 0 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd855b8f640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd85006ff20 0x7fd85014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd856390640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd85015a410 0x7fd8501f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.201+0000 7fd85538e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd856390640 10 monclient(hunting): get_auth_request con 0x7fd85015a410 auth_method 0 2026-02-22T23:26:31.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd856390640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd856390640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd856390640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd855b8f640 10 monclient(hunting): get_auth_request con 0x7fd85006ff20 auth_method 0 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd855b8f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd855b8f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd855b8f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd855b8f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd855b8f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd855b8f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd856390640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd856390640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.202+0000 7fd856390640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:31.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 10 monclient(hunting): handle_auth_done global_id 4908 payload 824 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 1 -- 10.20.193.6:0/401645665 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd85006ff20 msgr2=0x7fd85014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd85006ff20 0x7fd85014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 1 -- 10.20.193.6:0/401645665 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd85015a410 msgr2=0x7fd8501f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd85015a410 0x7fd8501f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 10 monclient: _finish_hunting 0 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 1 monclient: found mon.noname-a 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.203+0000 7fd85538e640 1 -- 10.20.193.6:0/401645665 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8501f45a0 con 0x7fd8500c99a0 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.205+0000 7fd85538e640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd8500c99a0 0x7fd8501f4ac0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd84400ebe0 tx=0x7fd84400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.205+0000 7fd83e7fc640 10 client.?.objecter ms_handle_connect 0x7fd8500c99a0 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.205+0000 7fd83e7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:31.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 1 -- 10.20.193.6:0/401645665 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844017070 con 0x7fd8500c99a0 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: dump: 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:31.220 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: _finish_auth 0 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 1 -- 10.20.193.6:0/401645665 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84400cc40 con 0x7fd8500c99a0 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 5 monclient: authenticate success, global_id 4908 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 10 monclient: _renew_subs 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8500c2510 con 0x7fd8500c99a0 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 1 -- 10.20.193.6:0/401645665 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844013470 con 0x7fd8500c99a0 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:31.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83e7fc640 10 monclient: dump: 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd83ffff640 4 set_mon_vals no callback set 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 10 client.4908.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:31.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 10 monclient: _renew_subs 2026-02-22T23:26:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8500c2ad0 con 0x7fd8500c99a0 2026-02-22T23:26:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.207+0000 7fd857e1a640 1 librados: init done 2026-02-22T23:26:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.208+0000 7fd857e1a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.208+0000 7fd857e1a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.208+0000 7fd857e1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:31.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.208+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd850070300 con 0x7fd8500c99a0 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.219+0000 7fd83e7fc640 1 -- 10.20.193.6:0/401645665 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd844024060 con 0x7fd8500c99a0 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.219+0000 7fd83e7fc640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd820078e00 0x7fd82007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.219+0000 7fd855b8f640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd820078e00 0x7fd82007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.220+0000 7fd855b8f640 10 monclient: get_auth_request con 0x7fd820078e00 auth_method 0 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.220+0000 7fd83e7fc640 10 client.4908.objecter ms_handle_connect 0x7fd820078e00 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.220+0000 7fd855b8f640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd820078e00 0x7fd82007b2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd8500c5380 tx=0x7fd840039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.221+0000 7fd83e7fc640 1 -- 10.20.193.6:0/401645665 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fd84405e220 con 0x7fd8500c99a0 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.221+0000 7fd83e7fc640 10 client.4908.objecter ms_dispatch2 0x7fd850000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:31.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.221+0000 7fd83e7fc640 3 client.4908.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:31.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.221+0000 7fd83e7fc640 3 client.4908.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:31.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.221+0000 7fd83e7fc640 20 client.4908.objecter dump_active .. 0 homeless 2026-02-22T23:26:31.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.318+0000 7fd83e7fc640 1 -- 10.20.193.6:0/401645665 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd8440621e0 con 0x7fd8500c99a0 2026-02-22T23:26:31.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.318+0000 7fd83e7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:31.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.318+0000 7fd83e7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:31.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.419+0000 7fd857e1a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:31.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.419+0000 7fd857e1a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:31.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.419+0000 7fd857e1a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:31.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.419+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd8501573f0 con 0x7fd8500c99a0 2026-02-22T23:26:31.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.423+0000 7fd83e7fc640 1 -- 10.20.193.6:0/401645665 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd8440664e0 con 0x7fd8500c99a0 2026-02-22T23:26:31.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.423+0000 7fd83e7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:31.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.423+0000 7fd83e7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:31.425 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:31.425 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 10 librados: watch_flush enter 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 10 librados: watch_flush exit 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 20 client.4908.objecter shutdown clearing up homeless session... 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 10 client.4908.objecter successfully canceled tick 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd820078e00 msgr2=0x7fd82007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd820078e00 0x7fd82007b2c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd8500c5380 tx=0x7fd840039040 comp rx=0 tx=0).stop 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 10 monclient: shutdown 2026-02-22T23:26:31.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd8500c99a0 msgr2=0x7fd8501f4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd8500c99a0 0x7fd8501f4ac0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd84400ebe0 tx=0x7fd84400c520 comp rx=0 tx=0).stop 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 shutdown_connections 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd820078e00 0x7fd82007b2c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd85015a410 0x7fd8501f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd8500c99a0 0x7fd8501f4ac0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 --2- 10.20.193.6:0/401645665 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd85006ff20 0x7fd85014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 >> 10.20.193.6:0/401645665 conn(0x7fd8500cd780 msgr2=0x7fd850157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 shutdown_connections 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 -- 10.20.193.6:0/401645665 wait complete. 2026-02-22T23:26:31.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:31.424+0000 7fd857e1a640 1 librados: shutdown 2026-02-22T23:26:33.244 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:17.493 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.245 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:17.491 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.245 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:17.495 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.246 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:17.493 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.253 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:17.503 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.254 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:17.500 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.271 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:17.482 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.271 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:17.483 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:33.279 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:17.491 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:34.436 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:34.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.505+0000 7f0db69d6640 10 monclient: get_monmap_and_config 2026-02-22T23:26:34.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.505+0000 7f0db69d6640 10 monclient: build_initial_monmap 2026-02-22T23:26:34.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.505+0000 7f0db69d6640 10 monclient: monmap: 2026-02-22T23:26:34.507 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:34.507 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:34.507 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:34.505854+0000 2026-02-22T23:26:34.507 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:34.505854+0000 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.505+0000 7f0db69d6640 1 Processor -- start 2026-02-22T23:26:34.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.505+0000 7f0db69d6640 1 -- start start 2026-02-22T23:26:34.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.505+0000 7f0db69d6640 10 monclient: init 2026-02-22T23:26:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient: _add_conns ranks=[0,2,5,4,1,3] 2026-02-22T23:26:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0db006ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0db00c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:26:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f0db015a5e0 0x7f0db0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient(hunting): picked mon.noname-c con 0x7f0db015a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db00c7e90 con 0x7f0db006ff20 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db00c8170 con 0x7f0db00c99a0 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0db0159a40 con 0x7f0db015a5e0 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db69d6640 10 monclient(hunting): authenticate will time out at 1269.233208s 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0daffff640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f0db006ff20 0x7f0db0159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58006/0 (socket says 10.20.193.6:58006) 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0daffff640 1 -- 10.20.193.6:0/4049640351 learned_addr learned my addr 10.20.193.6:0/4049640351 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.506+0000 7f0db4f4c640 1 --2- 10.20.193.6:0/4049640351 >> v2:10.20.193.80:3300/0 conn(0x7f0db015a5e0 0x7f0db0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0db4f4c640 10 monclient(hunting): get_auth_request con 0x7f0db015a5e0 auth_method 0 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0db4f4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:34.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0db4f4c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0db4f4c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0db4f4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0db4f4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0db4f4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0daeffd640 1 -- 10.20.193.6:0/4049640351 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1249404661 0 0) 0x7f0db00c7e90 con 0x7f0db006ff20 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0daeffd640 10 monclient(hunting): _negotiate 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0daeffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0daeffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0daeffd640 10 monclient(hunting): my global_id is 4911 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.507+0000 7f0daeffd640 1 -- 10.20.193.6:0/4049640351 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d94003620 con 0x7f0db006ff20 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 10 monclient(hunting): handle_auth_done global_id 4997 payload 274 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 1 -- 10.20.193.6:0/4049640351 >> v1:10.20.193.80:6789/0 conn(0x7f0db00c99a0 legacy=0x7f0db014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:34.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 1 -- 10.20.193.6:0/4049640351 >> v1:10.20.193.6:6789/0 conn(0x7f0db006ff20 legacy=0x7f0db0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:34.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 10 monclient: _finish_hunting 0 2026-02-22T23:26:34.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 1 monclient: found mon.noname-c 2026-02-22T23:26:34.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:34.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:26:34.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.508+0000 7f0db4f4c640 1 -- 10.20.193.6:0/4049640351 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db00c8450 con 0x7f0db015a5e0 2026-02-22T23:26:34.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.510+0000 7f0db4f4c640 1 --2- 10.20.193.6:0/4049640351 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db015a5e0 0x7f0db0070300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009dc0 tx=0x7f0da402f280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289b6421c128c322 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:34.519 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 1 -- 10.20.193.6:0/4049640351 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da403c070 con 0x7f0db015a5e0 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: dump: 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:34.520 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: _finish_auth 0 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: _check_auth_tickets 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 1 -- 10.20.193.6:0/4049640351 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da4037440 con 0x7f0db015a5e0 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0db69d6640 5 monclient: authenticate success, global_id 4997 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0db69d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 1 -- 10.20.193.6:0/4049640351 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4037810 con 0x7f0db015a5e0 2026-02-22T23:26:34.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0db69d6640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0db69d6640 4 set_mon_vals no callback set 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.512+0000 7f0daeffd640 10 monclient: dump: 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:34.522 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 10 monclient: shutdown 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 1 -- 10.20.193.6:0/4049640351 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db015a5e0 msgr2=0x7f0db0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 1 --2- 10.20.193.6:0/4049640351 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db015a5e0 0x7f0db0070300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009dc0 tx=0x7f0da402f280 comp rx=0 tx=0).stop 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 1 -- 10.20.193.6:0/4049640351 shutdown_connections 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 1 --2- 10.20.193.6:0/4049640351 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db015a5e0 0x7f0db0070300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 1 -- 10.20.193.6:0/4049640351 >> 10.20.193.6:0/4049640351 conn(0x7f0db00cd780 msgr2=0x7f0db00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 1 -- 10.20.193.6:0/4049640351 shutdown_connections 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 1 -- 10.20.193.6:0/4049640351 wait complete. 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 10 monclient: build_initial_monmap 2026-02-22T23:26:34.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.513+0000 7f0db69d6640 10 monclient: monmap: 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 librados: starting msgr at 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 librados: starting objecter 2026-02-22T23:26:34.524 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 Processor -- start 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 -- start start 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 librados: setting wanted keys 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 librados: calling monclient init 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient: init 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0db006ff20 0x7f0db014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): picked mon.noname-a con 0x7f0db006ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db00c99a0 0x7f0db006e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): picked mon.noname-c con 0x7f0db00c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0db015a5e0 0x7f0db006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:34.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): picked mon.noname-b con 0x7f0db015a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0150520 con 0x7f0db006ff20 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00c8a10 con 0x7f0db015a5e0 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01506a0 con 0x7f0db00c99a0 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.514+0000 7f0db69d6640 10 monclient(hunting): authenticate will time out at 1269.241813s 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0db006ff20 0x7f0db014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0db006ff20 0x7f0db014cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:44706/0 (socket says 10.20.193.6:44706) 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 1 -- 10.20.193.6:0/1985448453 learned_addr learned my addr 10.20.193.6:0/1985448453 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:34.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 10 monclient(hunting): get_auth_request con 0x7f0db006ff20 auth_method 0 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db00c99a0 0x7f0db006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 10 monclient(hunting): get_auth_request con 0x7f0db00c99a0 auth_method 0 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:34.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0db015a5e0 0x7f0db006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 10 monclient(hunting): get_auth_request con 0x7f0db015a5e0 auth_method 0 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.515+0000 7f0db4f4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 10 monclient(hunting): handle_auth_done global_id 5003 payload 824 2026-02-22T23:26:34.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 1 -- 10.20.193.6:0/1985448453 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0db015a5e0 msgr2=0x7f0db006e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0db015a5e0 0x7f0db006e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 1 -- 10.20.193.6:0/1985448453 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0db006ff20 msgr2=0x7f0db014cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0db006ff20 0x7f0db014cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 1 monclient: found mon.noname-c 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daf7fe640 1 -- 10.20.193.6:0/1985448453 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db01561b0 con 0x7f0db00c99a0 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.516+0000 7f0daffff640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0db006ff20 0x7f0db014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.517+0000 7f0daf7fe640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db00c99a0 0x7f0db006e470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0da000c4b0 tx=0x7f0da000c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.517+0000 7f0dacff9640 10 client.?.objecter ms_handle_connect 0x7f0db00c99a0 2026-02-22T23:26:34.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.517+0000 7f0dacff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:34.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 1 -- 10.20.193.6:0/1985448453 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da00052d0 con 0x7f0db00c99a0 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: dump: 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:34.532 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: _finish_auth 0 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: _check_auth_tickets 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 1 -- 10.20.193.6:0/1985448453 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da0005470 con 0x7f0db00c99a0 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 5 monclient: authenticate success, global_id 5003 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 1 -- 10.20.193.6:0/1985448453 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0003910 con 0x7f0db00c99a0 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dae7fc640 4 set_mon_vals no callback set 2026-02-22T23:26:34.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0dacff9640 10 monclient: dump: 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:34.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 10 monclient: _renew_subs 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db0156490 con 0x7f0db00c99a0 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 10 client.5003.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 10 monclient: _renew_subs 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db0156a50 con 0x7f0db00c99a0 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.519+0000 7f0db69d6640 1 librados: init done 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.520+0000 7f0db69d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.520+0000 7f0db69d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.520+0000 7f0db69d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:34.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.520+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db0070300 con 0x7f0db00c99a0 2026-02-22T23:26:34.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.531+0000 7f0dacff9640 1 -- 10.20.193.6:0/1985448453 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0da0003ab0 con 0x7f0db00c99a0 2026-02-22T23:26:34.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.531+0000 7f0dacff9640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0d80078e50 0x7f0d8007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:34.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.531+0000 7f0daffff640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0d80078e50 0x7f0d8007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:34.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.531+0000 7f0daffff640 10 monclient: get_auth_request con 0x7f0d80078e50 auth_method 0 2026-02-22T23:26:34.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.532+0000 7f0daffff640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0d80078e50 0x7f0d8007b310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c005980 tx=0x7f0d9c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:34.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.532+0000 7f0dacff9640 10 client.5003.objecter ms_handle_connect 0x7f0d80078e50 2026-02-22T23:26:34.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.627+0000 7f0dacff9640 1 -- 10.20.193.6:0/1985448453 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f0da005dfb0 con 0x7f0db00c99a0 2026-02-22T23:26:34.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.627+0000 7f0dacff9640 10 client.5003.objecter ms_dispatch2 0x7f0db0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:34.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.627+0000 7f0dacff9640 3 client.5003.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:34.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.627+0000 7f0dacff9640 3 client.5003.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:34.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.627+0000 7f0dacff9640 20 client.5003.objecter dump_active .. 0 homeless 2026-02-22T23:26:34.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.633+0000 7f0dacff9640 1 -- 10.20.193.6:0/1985448453 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0da0061f70 con 0x7f0db00c99a0 2026-02-22T23:26:34.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.633+0000 7f0dacff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:34.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.633+0000 7f0dacff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:34.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.733+0000 7f0db69d6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:34.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.733+0000 7f0db69d6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:34.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.733+0000 7f0db69d6640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:34.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.733+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0db01573f0 con 0x7f0db00c99a0 2026-02-22T23:26:34.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.736+0000 7f0dacff9640 1 -- 10.20.193.6:0/1985448453 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0da0066270 con 0x7f0db00c99a0 2026-02-22T23:26:34.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.736+0000 7f0dacff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:34.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.736+0000 7f0dacff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:34.738 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:34.738 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:34.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.736+0000 7f0db69d6640 10 librados: watch_flush enter 2026-02-22T23:26:34.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 10 librados: watch_flush exit 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 20 client.5003.objecter shutdown clearing up homeless session... 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 10 client.5003.objecter successfully canceled tick 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0d80078e50 msgr2=0x7f0d8007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0d80078e50 0x7f0d8007b310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c005980 tx=0x7f0d9c009040 comp rx=0 tx=0).stop 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 10 monclient: shutdown 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db00c99a0 msgr2=0x7f0db006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db00c99a0 0x7f0db006e470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0da000c4b0 tx=0x7f0da000c980 comp rx=0 tx=0).stop 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 shutdown_connections 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0d80078e50 0x7f0d8007b310 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:34.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0db015a5e0 0x7f0db006e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:34.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0db00c99a0 0x7f0db006e470 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:34.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 --2- 10.20.193.6:0/1985448453 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0db006ff20 0x7f0db014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:34.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 >> 10.20.193.6:0/1985448453 conn(0x7f0db00cd780 msgr2=0x7f0db00cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:34.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 shutdown_connections 2026-02-22T23:26:34.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 -- 10.20.193.6:0/1985448453 wait complete. 2026-02-22T23:26:34.741 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:34.737+0000 7f0db69d6640 1 librados: shutdown 2026-02-22T23:26:37.747 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.815+0000 7fd3f6ac9640 10 monclient: get_monmap_and_config 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.815+0000 7fd3f6ac9640 10 monclient: build_initial_monmap 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.815+0000 7fd3f6ac9640 10 monclient: monmap: 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:37.816122+0000 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:37.816122+0000 2026-02-22T23:26:37.817 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:37.818 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:37.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 1 Processor -- start 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 1 -- start start 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient: init 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient: _add_conns ranks=[2,0,1,5,3,4] 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd3f006ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd3f00c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd3f015a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f00c7e90 con 0x7fd3f00c99a0 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f00c8170 con 0x7fd3f015a5e0 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f00c8450 con 0x7fd3f006ff20 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:37.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3f6ac9640 10 monclient(hunting): authenticate will time out at 1272.543623s 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3e7fff640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fd3f00c99a0 0x7fd3f014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:58016/0 (socket says 10.20.193.6:58016) 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.816+0000 7fd3e7fff640 1 -- 10.20.193.6:0/2049904596 learned_addr learned my addr 10.20.193.6:0/2049904596 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003961398 0 0) 0x7fd3f00c7e90 con 0x7fd3f00c99a0 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _negotiate 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): my global_id is 4917 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3dc003620 con 0x7fd3f00c99a0 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483028147 0 0) 0x7fd3f00c8170 con 0x7fd3f015a5e0 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _negotiate 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): my global_id is 4879 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f00c7e90 con 0x7fd3f015a5e0 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145672265 0 0) 0x7fd3f00c8450 con 0x7fd3f006ff20 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _negotiate 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 10 monclient(hunting): my global_id is 5006 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.818+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f00c8170 con 0x7fd3f006ff20 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4180838633 0 0) 0x7fd3dc003620 con 0x7fd3f00c99a0 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 >> v1:10.20.193.80:6789/0 conn(0x7fd3f006ff20 legacy=0x7fd3f0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 >> v1:10.20.193.40:6789/0 conn(0x7fd3f015a5e0 legacy=0x7fd3f00c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:37.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: _finish_hunting 0 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f00c8730 con 0x7fd3f00c99a0 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: _finish_auth 0 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fd3d8002f00 con 0x7fd3f00c99a0 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3f6ac9640 5 monclient: authenticate success, global_id 4917 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:26:37.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: dump: 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:37.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 >> v1:10.20.193.6:6789/0 conn(0x7fd3f00c99a0 legacy=0x7fd3f014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3dc003fe0 0x7fd3dc004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): picked mon.a con 0x7fd3dc003fe0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3dc004e40 0x7fd3dc0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): picked mon.c con 0x7fd3dc004e40 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc009ab0 0x7fd3dc00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): picked mon.b con 0x7fd3dc009ab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc00c570 con 0x7fd3dc003fe0 2026-02-22T23:26:37.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc00c6f0 con 0x7fd3dc009ab0 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc00c8f0 con 0x7fd3dc004e40 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3e77fe640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.819+0000 7fd3f6ac9640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.820+0000 7fd3e7fff640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3dc004e40 0x7fd3dc0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.820+0000 7fd3f503f640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc009ab0 0x7fd3dc00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f503f640 10 monclient(hunting): get_auth_request con 0x7fd3dc009ab0 auth_method 0 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f503f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f503f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f503f640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3e7fff640 10 monclient(hunting): get_auth_request con 0x7fd3dc004e40 auth_method 0 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3e7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f503f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f503f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f503f640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3dc003fe0 0x7fd3dc004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:37.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 10 monclient(hunting): get_auth_request con 0x7fd3dc003fe0 auth_method 0 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.821+0000 7fd3f483e640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 10 monclient(hunting): handle_auth_done global_id 4917 payload 293 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 1 -- 10.20.193.6:0/2049904596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc009ab0 msgr2=0x7fd3dc00bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc009ab0 0x7fd3dc00bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 1 -- 10.20.193.6:0/2049904596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3dc003fe0 msgr2=0x7fd3dc004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:37.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3dc003fe0 0x7fd3dc004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 1 monclient: found mon.c 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 1 -- 10.20.193.6:0/2049904596 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc003620 con 0x7fd3dc004e40 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 10 monclient: _finish_auth 0 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.822+0000 7fd3f503f640 1 -- 10.20.193.6:0/2049904596 reap_dead start 2026-02-22T23:26:37.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.823+0000 7fd3e7fff640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3dc004e40 0x7fd3dc0052c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c1c0 tx=0x7fd3d802f540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=970fe9d0cf41efe9 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:37.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8007d00 con 0x7fd3dc004e40 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 10 monclient: dump: 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:37.841 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3d8005bf0 con 0x7fd3dc004e40 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3f6ac9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 1 -- 10.20.193.6:0/2049904596 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8007070 con 0x7fd3dc004e40 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3e77fe640 10 monclient: dump: 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:37.842 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3f6ac9640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3f6ac9640 4 set_mon_vals no callback set 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3f6ac9640 10 monclient: shutdown 2026-02-22T23:26:37.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3f6ac9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/2049904596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3dc004e40 msgr2=0x7fd3dc0052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.824+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3dc004e40 0x7fd3dc0052c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c1c0 tx=0x7fd3d802f540 comp rx=0 tx=0).stop 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/2049904596 shutdown_connections 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/2049904596 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3dc004e40 0x7fd3dc0052c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/2049904596 >> 10.20.193.6:0/2049904596 conn(0x7fd3f00cd780 msgr2=0x7fd3f00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/2049904596 shutdown_connections 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/2049904596 wait complete. 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 10 monclient: build_initial_monmap 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 10 monclient: monmap: 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:37.844 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 librados: starting msgr at 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 librados: starting objecter 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 Processor -- start 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 -- start start 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 librados: setting wanted keys 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 1 librados: calling monclient init 2026-02-22T23:26:37.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.825+0000 7fd3f6ac9640 10 monclient: init 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc004e40 0x7fd3f0148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): picked mon.noname-b con 0x7fd3dc004e40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3f01493d0 0x7fd3f01f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): picked mon.noname-c con 0x7fd3f01493d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3f01f63e0 0x7fd3f00cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): picked mon.noname-a con 0x7fd3f01f63e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f0070300 con 0x7fd3f01f63e0 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f0159a40 con 0x7fd3dc004e40 2026-02-22T23:26:37.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f0070480 con 0x7fd3f01493d0 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f6ac9640 10 monclient(hunting): authenticate will time out at 1272.553144s 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3f01f63e0 0x7fd3f00cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3f01f63e0 0x7fd3f00cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:44720/0 (socket says 10.20.193.6:44720) 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 1 -- 10.20.193.6:0/469115177 learned_addr learned my addr 10.20.193.6:0/469115177 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 10 monclient(hunting): get_auth_request con 0x7fd3f01f63e0 auth_method 0 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3f01493d0 0x7fd3f01f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:37.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f483e640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc004e40 0x7fd3f0148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 10 monclient(hunting): get_auth_request con 0x7fd3f01493d0 auth_method 0 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f503f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f483e640 10 monclient(hunting): get_auth_request con 0x7fd3dc004e40 auth_method 0 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f483e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f483e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3f483e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:37.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.826+0000 7fd3e7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.827+0000 7fd3f483e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.827+0000 7fd3f483e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.827+0000 7fd3f483e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 10 monclient(hunting): handle_auth_done global_id 4920 payload 824 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 1 -- 10.20.193.6:0/469115177 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3f01493d0 msgr2=0x7fd3f01f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3f01493d0 0x7fd3f01f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 1 -- 10.20.193.6:0/469115177 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc004e40 msgr2=0x7fd3f0148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc004e40 0x7fd3f0148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 10 monclient: _finish_hunting 0 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 1 monclient: found mon.noname-a 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:37.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.828+0000 7fd3f503f640 1 -- 10.20.193.6:0/469115177 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f0156c50 con 0x7fd3f01f63e0 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.829+0000 7fd3f503f640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3f01f63e0 0x7fd3f00cc1c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3f015a9c0 tx=0x7fd3ec00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.829+0000 7fd3e57fa640 10 client.?.objecter ms_handle_connect 0x7fd3f01f63e0 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.829+0000 7fd3e57fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 1 -- 10.20.193.6:0/469115177 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec019070 con 0x7fd3f01f63e0 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: dump: 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.850 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: _finish_auth 0 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: _check_auth_tickets 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 1 -- 10.20.193.6:0/469115177 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ec00fda0 con 0x7fd3f01f63e0 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 5 monclient: authenticate success, global_id 4920 2026-02-22T23:26:37.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 10 monclient: _renew_subs 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f0156f30 con 0x7fd3f01f63e0 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 1 -- 10.20.193.6:0/469115177 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec004270 con 0x7fd3f01f63e0 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e6ffd640 4 set_mon_vals no callback set 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3e57fa640 10 monclient: dump: 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:37.852 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 10 client.4920.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 10 monclient: _renew_subs 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f01f56a0 con 0x7fd3f01f63e0 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.830+0000 7fd3f6ac9640 1 librados: init done 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.831+0000 7fd3f6ac9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.831+0000 7fd3f6ac9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:37.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.831+0000 7fd3f6ac9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:37.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.831+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f0159ed0 con 0x7fd3f01f63e0 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.845+0000 7fd3e57fa640 1 -- 10.20.193.6:0/469115177 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd3ec004410 con 0x7fd3f01f63e0 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.845+0000 7fd3e57fa640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd3c0078e80 0x7fd3c007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.845+0000 7fd3f483e640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd3c0078e80 0x7fd3c007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.845+0000 7fd3f483e640 10 monclient: get_auth_request con 0x7fd3c0078e80 auth_method 0 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.846+0000 7fd3f483e640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd3c0078e80 0x7fd3c007b340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3f015de10 tx=0x7fd3e0005360 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.846+0000 7fd3e57fa640 10 client.4920.objecter ms_handle_connect 0x7fd3c0078e80 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.852+0000 7fd3e57fa640 1 -- 10.20.193.6:0/469115177 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fd3ec05dd90 con 0x7fd3f01f63e0 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.852+0000 7fd3e57fa640 10 client.4920.objecter ms_dispatch2 0x7fd3f0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.852+0000 7fd3e57fa640 3 client.4920.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:37.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.852+0000 7fd3e57fa640 3 client.4920.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:37.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.852+0000 7fd3e57fa640 20 client.4920.objecter dump_active .. 0 homeless 2026-02-22T23:26:37.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.941+0000 7fd3e57fa640 1 -- 10.20.193.6:0/469115177 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd3ec024020 con 0x7fd3f01f63e0 2026-02-22T23:26:37.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.941+0000 7fd3e57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:37.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:37.941+0000 7fd3e57fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:38.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.044+0000 7fd3f6ac9640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:38.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.044+0000 7fd3f6ac9640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:38.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.044+0000 7fd3f6ac9640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:38.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.044+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd3f01573f0 con 0x7fd3f01f63e0 2026-02-22T23:26:38.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.046+0000 7fd3e57fa640 1 -- 10.20.193.6:0/469115177 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd3ec061d50 con 0x7fd3f01f63e0 2026-02-22T23:26:38.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.046+0000 7fd3e57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:38.048 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.046+0000 7fd3e57fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:38.048 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:38.048 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:38.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 10 librados: watch_flush enter 2026-02-22T23:26:38.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 10 librados: watch_flush exit 2026-02-22T23:26:38.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 20 client.4920.objecter shutdown clearing up homeless session... 2026-02-22T23:26:38.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 10 client.4920.objecter successfully canceled tick 2026-02-22T23:26:38.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd3c0078e80 msgr2=0x7fd3c007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:38.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd3c0078e80 0x7fd3c007b340 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3f015de10 tx=0x7fd3e0005360 comp rx=0 tx=0).stop 2026-02-22T23:26:38.050 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 10 monclient: shutdown 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3f01f63e0 msgr2=0x7fd3f00cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3f01f63e0 0x7fd3f00cc1c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3f015a9c0 tx=0x7fd3ec00f5a0 comp rx=0 tx=0).stop 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 shutdown_connections 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd3c0078e80 0x7fd3c007b340 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd3f01f63e0 0x7fd3f00cc1c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd3f01493d0 0x7fd3f01f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 --2- 10.20.193.6:0/469115177 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd3dc004e40 0x7fd3f0148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 >> 10.20.193.6:0/469115177 conn(0x7fd3f00cd780 msgr2=0x7fd3f00cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 shutdown_connections 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 -- 10.20.193.6:0/469115177 wait complete. 2026-02-22T23:26:38.051 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:38.047+0000 7fd3f6ac9640 1 librados: shutdown 2026-02-22T23:26:41.059 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:41.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.125+0000 7fb670669640 10 monclient: get_monmap_and_config 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.125+0000 7fb670669640 10 monclient: build_initial_monmap 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.125+0000 7fb670669640 10 monclient: monmap: 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:41.126415+0000 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:41.126415+0000 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:41.128 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:41.129 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:41.129 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:41.129 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:41.129 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:41.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 1 Processor -- start 2026-02-22T23:26:41.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 1 -- start start 2026-02-22T23:26:41.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient: init 2026-02-22T23:26:41.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient: _add_conns ranks=[2,4,3,5,1,0] 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb6681527d0 addr v1:10.20.193.80:6789/0 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fb668152bb0 0x7fb6680c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient(hunting): picked mon.noname-b con 0x7fb668152bb0 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fb6680c5fc0 0x7fb6680c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient(hunting): picked mon.noname-a con 0x7fb6680c5fc0 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66806e300 con 0x7fb6681527d0 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fb66806f1f0 con 0x7fb6680c5fc0 2026-02-22T23:26:41.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb6680c6470 con 0x7fb668152bb0 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.126+0000 7fb670669640 10 monclient(hunting): authenticate will time out at 1275.853749s 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fb668152bb0 0x7fb6680c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fb6680c5fc0 0x7fb6680c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66e3de640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fb6681527d0 0x7fb66806da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:33854/0 (socket says 10.20.193.6:33854) 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66e3de640 1 -- 10.20.193.6:0/1064703627 learned_addr learned my addr 10.20.193.6:0/1064703627 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 10 monclient(hunting): get_auth_request con 0x7fb668152bb0 auth_method 0 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:41.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 10 monclient(hunting): get_auth_request con 0x7fb6680c5fc0 auth_method 0 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.127+0000 7fb66ebdf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66d3dc640 1 -- 10.20.193.6:0/1064703627 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216645791 0 0) 0x7fb66806e300 con 0x7fb6681527d0 2026-02-22T23:26:41.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66d3dc640 10 monclient(hunting): _negotiate 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66d3dc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66d3dc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66d3dc640 10 monclient(hunting): my global_id is 5012 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66d3dc640 1 -- 10.20.193.6:0/1064703627 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6500058b0 con 0x7fb6681527d0 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_done global_id 4885 payload 274 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 1 -- 10.20.193.6:0/1064703627 >> v2:10.20.193.6:3300/0 conn(0x7fb6680c5fc0 msgr2=0x7fb6680c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 1 --2- 10.20.193.6:0/1064703627 >> v2:10.20.193.6:3300/0 conn(0x7fb6680c5fc0 0x7fb6680c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 1 -- 10.20.193.6:0/1064703627 >> v1:10.20.193.80:6789/0 conn(0x7fb6681527d0 legacy=0x7fb66806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 10 monclient: _finish_hunting 0 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 1 monclient: found mon.noname-b 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:41.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.128+0000 7fb66dbdd640 1 -- 10.20.193.6:0/1064703627 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66806e5e0 con 0x7fb668152bb0 2026-02-22T23:26:41.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.130+0000 7fb66dbdd640 1 --2- 10.20.193.6:0/1064703627 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb668152bb0 0x7fb6680c5a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6680750e0 tx=0x7fb65c02ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a0ede83842bf5ab server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:41.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 1 -- 10.20.193.6:0/1064703627 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c03c070 con 0x7fb668152bb0 2026-02-22T23:26:41.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:41.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:41.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: dump: 2026-02-22T23:26:41.142 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:41.142 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: _finish_auth 0 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: _check_auth_tickets 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:41.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 1 -- 10.20.193.6:0/1064703627 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65c02faf0 con 0x7fb668152bb0 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 1 -- 10.20.193.6:0/1064703627 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c037440 con 0x7fb668152bb0 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb670669640 5 monclient: authenticate success, global_id 4885 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb66d3dc640 10 monclient: dump: 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:41.144 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb670669640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb670669640 4 set_mon_vals no callback set 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb670669640 10 monclient: shutdown 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb670669640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.131+0000 7fb670669640 1 -- 10.20.193.6:0/1064703627 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb668152bb0 msgr2=0x7fb6680c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 --2- 10.20.193.6:0/1064703627 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb668152bb0 0x7fb6680c5a80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6680750e0 tx=0x7fb65c02ede0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 -- 10.20.193.6:0/1064703627 shutdown_connections 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 --2- 10.20.193.6:0/1064703627 >> v2:10.20.193.6:3300/0 conn(0x7fb6680c5fc0 0x7fb6680c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 --2- 10.20.193.6:0/1064703627 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb668152bb0 0x7fb6680c5a80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 -- 10.20.193.6:0/1064703627 >> 10.20.193.6:0/1064703627 conn(0x7fb6680cd780 msgr2=0x7fb6680cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 -- 10.20.193.6:0/1064703627 shutdown_connections 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 -- 10.20.193.6:0/1064703627 wait complete. 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 10 monclient: build_initial_monmap 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 10 monclient: monmap: 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:41.146 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 librados: starting msgr at 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 librados: starting objecter 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 Processor -- start 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 -- start start 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 librados: setting wanted keys 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 1 librados: calling monclient init 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.132+0000 7fb670669640 10 monclient: init 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb6680c5fc0 0x7fb6680ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): picked mon.noname-a con 0x7fb6680c5fc0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb6681527d0 0x7fb668147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): picked mon.noname-b con 0x7fb6681527d0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb668152bb0 0x7fb6680bc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): picked mon.noname-c con 0x7fb668152bb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb66806ee80 con 0x7fb6680c5fc0 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb66806eba0 con 0x7fb6681527d0 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680c93a0 con 0x7fb668152bb0 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb670669640 10 monclient(hunting): authenticate will time out at 1275.860385s 2026-02-22T23:26:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66e3de640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb6680c5fc0 0x7fb6680ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66dbdd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb6681527d0 0x7fb668147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66e3de640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb6680c5fc0 0x7fb6680ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54768/0 (socket says 10.20.193.6:54768) 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66e3de640 1 -- 10.20.193.6:0/3886724083 learned_addr learned my addr 10.20.193.6:0/3886724083 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66e3de640 10 monclient(hunting): get_auth_request con 0x7fb6680c5fc0 auth_method 0 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66e3de640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66e3de640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66e3de640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66ebdf640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb668152bb0 0x7fb6680bc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66dbdd640 10 monclient(hunting): get_auth_request con 0x7fb6681527d0 auth_method 0 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66dbdd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66dbdd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:41.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.133+0000 7fb66dbdd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66ebdf640 10 monclient(hunting): get_auth_request con 0x7fb668152bb0 auth_method 0 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66ebdf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66ebdf640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66ebdf640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66e3de640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66e3de640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66e3de640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66ebdf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66ebdf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:41.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.134+0000 7fb66ebdf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 10 monclient(hunting): handle_auth_done global_id 4888 payload 824 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 1 -- 10.20.193.6:0/3886724083 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb668152bb0 msgr2=0x7fb6680bc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb668152bb0 0x7fb6680bc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 1 -- 10.20.193.6:0/3886724083 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb6680c5fc0 msgr2=0x7fb6680ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb6680c5fc0 0x7fb6680ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 10 monclient: _finish_hunting 0 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 1 monclient: found mon.noname-b 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.135+0000 7fb66dbdd640 1 -- 10.20.193.6:0/3886724083 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6681f33e0 con 0x7fb6681527d0 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.136+0000 7fb66dbdd640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb6681527d0 0x7fb668147c90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb65c036000 tx=0x7fb65c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:41.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.136+0000 7fb64effd640 10 client.?.objecter ms_handle_connect 0x7fb6681527d0 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.136+0000 7fb64effd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 1 -- 10.20.193.6:0/3886724083 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c03c040 con 0x7fb6681527d0 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: dump: 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:41.152 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: _finish_auth 0 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: _check_auth_tickets 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 1 -- 10.20.193.6:0/3886724083 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65c042450 con 0x7fb6681527d0 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 5 monclient: authenticate success, global_id 4888 2026-02-22T23:26:41.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 monclient: _renew_subs 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6681ef130 con 0x7fb6681527d0 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 1 -- 10.20.193.6:0/3886724083 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c042820 con 0x7fb6681527d0 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb64effd640 10 monclient: dump: 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:41.154 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb66cbdb640 4 set_mon_vals no callback set 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 client.4888.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 monclient: _renew_subs 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6681ef6f0 con 0x7fb6681527d0 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 1 librados: init done 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:41.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:41.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.138+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6681efd00 con 0x7fb6681527d0 2026-02-22T23:26:41.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.149+0000 7fb64effd640 1 -- 10.20.193.6:0/3886724083 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb65c0429c0 con 0x7fb6681527d0 2026-02-22T23:26:41.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.150+0000 7fb64effd640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb638078e30 0x7fb63807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:41.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.150+0000 7fb66e3de640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb638078e30 0x7fb63807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.150+0000 7fb66e3de640 10 monclient: get_auth_request con 0x7fb638078e30 auth_method 0 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.151+0000 7fb66e3de640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb638078e30 0x7fb63807b2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb658005980 tx=0x7fb658009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.151+0000 7fb64effd640 10 client.4888.objecter ms_handle_connect 0x7fb638078e30 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.151+0000 7fb64effd640 1 -- 10.20.193.6:0/3886724083 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fb65c02f9b0 con 0x7fb6681527d0 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.151+0000 7fb64effd640 10 client.4888.objecter ms_dispatch2 0x7fb668000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.151+0000 7fb64effd640 3 client.4888.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.151+0000 7fb64effd640 3 client.4888.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:41.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.151+0000 7fb64effd640 20 client.4888.objecter dump_active .. 0 homeless 2026-02-22T23:26:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.255+0000 7fb64effd640 1 -- 10.20.193.6:0/3886724083 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb65c0873c0 con 0x7fb6681527d0 2026-02-22T23:26:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.255+0000 7fb64effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.255+0000 7fb64effd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:41.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.362+0000 7fb670669640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:41.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.362+0000 7fb670669640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:41.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.362+0000 7fb670669640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:26:41.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.362+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb6680c9730 con 0x7fb6681527d0 2026-02-22T23:26:41.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.365+0000 7fb64effd640 1 -- 10.20.193.6:0/3886724083 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb65c08b6c0 con 0x7fb6681527d0 2026-02-22T23:26:41.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.365+0000 7fb64effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:41.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.365+0000 7fb64effd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:41.367 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:41.367 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 10 librados: watch_flush enter 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 10 librados: watch_flush exit 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 20 client.4888.objecter shutdown clearing up homeless session... 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 10 client.4888.objecter successfully canceled tick 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb638078e30 msgr2=0x7fb63807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb638078e30 0x7fb63807b2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb658005980 tx=0x7fb658009040 comp rx=0 tx=0).stop 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 10 monclient: shutdown 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb6681527d0 msgr2=0x7fb668147c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb6681527d0 0x7fb668147c90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb65c036000 tx=0x7fb65c0047c0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 shutdown_connections 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb638078e30 0x7fb63807b2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb668152bb0 0x7fb6680bc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb6681527d0 0x7fb668147c90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 --2- 10.20.193.6:0/3886724083 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb6680c5fc0 0x7fb6680ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 >> 10.20.193.6:0/3886724083 conn(0x7fb6680cd780 msgr2=0x7fb6681504d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 shutdown_connections 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 -- 10.20.193.6:0/3886724083 wait complete. 2026-02-22T23:26:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:41.366+0000 7fb670669640 1 librados: shutdown 2026-02-22T23:26:44.378 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:44.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.444+0000 7f836439e640 10 monclient: get_monmap_and_config 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.444+0000 7f836439e640 10 monclient: build_initial_monmap 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.444+0000 7f836439e640 10 monclient: monmap: 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:44.445307+0000 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:44.445307+0000 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:44.446 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:44.447 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:44.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 1 Processor -- start 2026-02-22T23:26:44.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 1 -- start start 2026-02-22T23:26:44.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient: init 2026-02-22T23:26:44.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient: _add_conns ranks=[3,5,1,0,4,2] 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f835c06ff20 0x7f835c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient(hunting): picked mon.noname-a con 0x7f835c06ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f835c0c99a0 0x7f835c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient(hunting): picked mon.noname-c con 0x7f835c0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f835c15a410 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f835c15a860 con 0x7f835c15a410 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f835c159a40 con 0x7f835c06ff20 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f835c0c7e90 con 0x7f835c0c99a0 2026-02-22T23:26:44.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.445+0000 7f836439e640 10 monclient(hunting): authenticate will time out at 1279.172763s 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f835c06ff20 0x7f835c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f835c0c99a0 0x7f835c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362914640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f835c15a410 0x7f835c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:54474/0 (socket says 10.20.193.6:54474) 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362914640 1 -- 10.20.193.6:0/3346371417 learned_addr learned my addr 10.20.193.6:0/3346371417 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 10 monclient(hunting): get_auth_request con 0x7f835c06ff20 auth_method 0 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 10 monclient(hunting): get_auth_request con 0x7f835c0c99a0 auth_method 0 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:44.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8362113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.446+0000 7f8361912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 10 monclient(hunting): handle_auth_done global_id 5018 payload 274 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 1 -- 10.20.193.6:0/3346371417 >> v2:10.20.193.6:3300/0 conn(0x7f835c06ff20 msgr2=0x7f835c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 1 --2- 10.20.193.6:0/3346371417 >> v2:10.20.193.6:3300/0 conn(0x7f835c06ff20 0x7f835c070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 1 -- 10.20.193.6:0/3346371417 >> v1:10.20.193.40:6789/0 conn(0x7f835c15a410 legacy=0x7f835c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 10 monclient: _finish_hunting 0 2026-02-22T23:26:44.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 1 monclient: found mon.noname-c 2026-02-22T23:26:44.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:44.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:26:44.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.447+0000 7f8361912640 1 -- 10.20.193.6:0/3346371417 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0c8170 con 0x7f835c0c99a0 2026-02-22T23:26:44.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.448+0000 7f8361912640 1 --2- 10.20.193.6:0/3346371417 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c0c99a0 0x7f835c159ed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f83500098a0 tx=0x7f835002ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ebea254f02ed9f2 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:44.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 1 -- 10.20.193.6:0/3346371417 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835003c070 con 0x7f835c0c99a0 2026-02-22T23:26:44.459 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: dump: 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:44.460 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: _finish_auth 0 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: _check_auth_tickets 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 1 -- 10.20.193.6:0/3346371417 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835002fae0 con 0x7f835c0c99a0 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 1 -- 10.20.193.6:0/3346371417 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350037440 con 0x7f835c0c99a0 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f8361111640 10 monclient: dump: 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:44.461 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 5 monclient: authenticate success, global_id 5018 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 4 set_mon_vals no callback set 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 10 monclient: shutdown 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:44.462 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 -- 10.20.193.6:0/3346371417 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c0c99a0 msgr2=0x7f835c159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 --2- 10.20.193.6:0/3346371417 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c0c99a0 0x7f835c159ed0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f83500098a0 tx=0x7f835002ee10 comp rx=0 tx=0).stop 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 -- 10.20.193.6:0/3346371417 shutdown_connections 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 --2- 10.20.193.6:0/3346371417 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c0c99a0 0x7f835c159ed0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 --2- 10.20.193.6:0/3346371417 >> v2:10.20.193.6:3300/0 conn(0x7f835c06ff20 0x7f835c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 -- 10.20.193.6:0/3346371417 >> 10.20.193.6:0/3346371417 conn(0x7f835c0cd780 msgr2=0x7f835c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 -- 10.20.193.6:0/3346371417 shutdown_connections 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.450+0000 7f836439e640 1 -- 10.20.193.6:0/3346371417 wait complete. 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient: build_initial_monmap 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient: monmap: 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:44.463 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 librados: starting msgr at 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 librados: starting objecter 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 Processor -- start 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 -- start start 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 librados: setting wanted keys 2026-02-22T23:26:44.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 librados: calling monclient init 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient: init 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f835c06ff20 0x7f835c1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): picked mon.noname-b con 0x7f835c06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f835c0c99a0 0x7f835c14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): picked mon.noname-a con 0x7f835c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c15a410 0x7f835c1f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): picked mon.noname-c con 0x7f835c15a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0c8a10 con 0x7f835c0c99a0 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:44.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f835c0c8730 con 0x7f835c06ff20 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f835c150520 con 0x7f835c15a410 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.451+0000 7f836439e640 10 monclient(hunting): authenticate will time out at 1279.178945s 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c15a410 0x7f835c1f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f835c06ff20 0x7f835c1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f835c0c99a0 0x7f835c14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c15a410 0x7f835c1f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:46812/0 (socket says 10.20.193.6:46812) 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 1 -- 10.20.193.6:0/3563257403 learned_addr learned my addr 10.20.193.6:0/3563257403 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 10 monclient(hunting): get_auth_request con 0x7f835c15a410 auth_method 0 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:44.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 10 monclient(hunting): get_auth_request con 0x7f835c0c99a0 auth_method 0 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 10 monclient(hunting): get_auth_request con 0x7f835c06ff20 auth_method 0 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362914640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:44.467 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8362113640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.452+0000 7f8361912640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 10 monclient(hunting): handle_auth_done global_id 5021 payload 824 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 1 -- 10.20.193.6:0/3563257403 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f835c06ff20 msgr2=0x7f835c1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f835c06ff20 0x7f835c1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 1 -- 10.20.193.6:0/3563257403 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f835c0c99a0 msgr2=0x7f835c14bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f835c0c99a0 0x7f835c14bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 10 monclient: _finish_hunting 0 2026-02-22T23:26:44.468 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 1 monclient: found mon.noname-c 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.453+0000 7f8362914640 1 -- 10.20.193.6:0/3563257403 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c0c1f90 con 0x7f835c15a410 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.454+0000 7f8362914640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c15a410 0x7f835c1f4e40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f835800cad0 tx=0x7f835800a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.454+0000 7f8342ffd640 10 client.?.objecter ms_handle_connect 0x7f835c15a410 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.454+0000 7f8342ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 1 -- 10.20.193.6:0/3563257403 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358017070 con 0x7f835c15a410 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: dump: 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:44.469 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: _finish_auth 0 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 1 -- 10.20.193.6:0/3563257403 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835800ac70 con 0x7f835c15a410 2026-02-22T23:26:44.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 1 -- 10.20.193.6:0/3563257403 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358012640 con 0x7f835c15a410 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 5 monclient: authenticate success, global_id 5021 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8343fff640 4 set_mon_vals no callback set 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f8342ffd640 10 monclient: dump: 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:44.471 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 10 monclient: _renew_subs 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c0c2270 con 0x7f835c15a410 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 10 client.5021.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 10 monclient: _renew_subs 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c0c2830 con 0x7f835c15a410 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.456+0000 7f836439e640 1 librados: init done 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.457+0000 7f836439e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:44.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.457+0000 7f836439e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:44.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.457+0000 7f836439e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:44.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.457+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c070300 con 0x7f835c15a410 2026-02-22T23:26:44.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.468+0000 7f8342ffd640 1 -- 10.20.193.6:0/3563257403 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f835801b3b0 con 0x7f835c15a410 2026-02-22T23:26:44.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.468+0000 7f8342ffd640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f832c078e50 0x7f832c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:44.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.469+0000 7f8362113640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f832c078e50 0x7f832c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:44.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.469+0000 7f8362113640 10 monclient: get_auth_request con 0x7f832c078e50 auth_method 0 2026-02-22T23:26:44.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.469+0000 7f8362113640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f832c078e50 0x7f832c07b310 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f834c009c40 tx=0x7f834c009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:44.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.469+0000 7f8342ffd640 10 client.5021.objecter ms_handle_connect 0x7f832c078e50 2026-02-22T23:26:44.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.560+0000 7f8342ffd640 1 -- 10.20.193.6:0/3563257403 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f835805dc60 con 0x7f835c15a410 2026-02-22T23:26:44.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.560+0000 7f8342ffd640 10 client.5021.objecter ms_dispatch2 0x7f835c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:44.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.560+0000 7f8342ffd640 3 client.5021.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:44.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.560+0000 7f8342ffd640 3 client.5021.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:44.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.560+0000 7f8342ffd640 20 client.5021.objecter dump_active .. 0 homeless 2026-02-22T23:26:44.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.566+0000 7f8342ffd640 1 -- 10.20.193.6:0/3563257403 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8358021020 con 0x7f835c15a410 2026-02-22T23:26:44.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.566+0000 7f8342ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:44.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.566+0000 7f8342ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:44.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.666+0000 7f836439e640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:44.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.666+0000 7f836439e640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:44.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.666+0000 7f836439e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:44.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.666+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f835c1f4ba0 con 0x7f835c15a410 2026-02-22T23:26:44.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.668+0000 7f8342ffd640 1 -- 10.20.193.6:0/3563257403 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f835800ae20 con 0x7f835c15a410 2026-02-22T23:26:44.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.668+0000 7f8342ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:44.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.668+0000 7f8342ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:44.670 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:44.670 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:44.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 10 librados: watch_flush enter 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 10 librados: watch_flush exit 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 20 client.5021.objecter shutdown clearing up homeless session... 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 10 client.5021.objecter successfully canceled tick 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f832c078e50 msgr2=0x7f832c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f832c078e50 0x7f832c07b310 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f834c009c40 tx=0x7f834c009290 comp rx=0 tx=0).stop 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 10 monclient: shutdown 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c15a410 msgr2=0x7f835c1f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c15a410 0x7f835c1f4e40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f835800cad0 tx=0x7f835800a060 comp rx=0 tx=0).stop 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 shutdown_connections 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f832c078e50 0x7f832c07b310 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f835c15a410 0x7f835c1f4e40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f835c0c99a0 0x7f835c14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 --2- 10.20.193.6:0/3563257403 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f835c06ff20 0x7f835c1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:44.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 >> 10.20.193.6:0/3563257403 conn(0x7f835c0cd780 msgr2=0x7f835c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:44.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 shutdown_connections 2026-02-22T23:26:44.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 -- 10.20.193.6:0/3563257403 wait complete. 2026-02-22T23:26:44.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:44.669+0000 7f836439e640 1 librados: shutdown 2026-02-22T23:26:47.679 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:47.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.746+0000 7f1c10987640 10 monclient: get_monmap_and_config 2026-02-22T23:26:47.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.746+0000 7f1c10987640 10 monclient: build_initial_monmap 2026-02-22T23:26:47.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.746+0000 7f1c10987640 10 monclient: monmap: 2026-02-22T23:26:47.748 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:47.748 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:47.748 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:47.746954+0000 2026-02-22T23:26:47.748 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:47.746954+0000 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.746+0000 7f1c10987640 1 Processor -- start 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.746+0000 7f1c10987640 1 -- start start 2026-02-22T23:26:47.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.746+0000 7f1c10987640 10 monclient: init 2026-02-22T23:26:47.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:47.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient: _add_conns ranks=[3,0,1,2,4,5] 2026-02-22T23:26:47.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1c0806ff20 0x7f1c08070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient(hunting): picked mon.noname-a con 0x7f1c0806ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f1c080c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1c0815a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c080c7e90 con 0x7f1c080c99a0 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c080c8170 con 0x7f1c0815a5e0 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1c08159a40 con 0x7f1c0806ff20 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c10987640 10 monclient(hunting): authenticate will time out at 1282.474301s 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c0eefd640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f1c0815a5e0 0x7f1c0814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:54482/0 (socket says 10.20.193.6:54482) 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c0eefd640 1 -- 10.20.193.6:0/1845039553 learned_addr learned my addr 10.20.193.6:0/1845039553 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:47.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.747+0000 7f1c0e6fc640 1 --2- 10.20.193.6:0/1845039553 >> v2:10.20.193.6:3300/0 conn(0x7f1c0806ff20 0x7f1c08070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.748+0000 7f1c0e6fc640 10 monclient(hunting): get_auth_request con 0x7f1c0806ff20 auth_method 0 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.748+0000 7f1c0e6fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.748+0000 7f1c0e6fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.748+0000 7f1c0e6fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.748+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.748+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.748+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0d6fa640 1 -- 10.20.193.6:0/1845039553 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739897936 0 0) 0x7f1c080c8170 con 0x7f1c0815a5e0 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0d6fa640 10 monclient(hunting): _negotiate 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0d6fa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0d6fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:47.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0d6fa640 10 monclient(hunting): my global_id is 4897 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0d6fa640 1 -- 10.20.193.6:0/1845039553 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bf8003620 con 0x7f1c0815a5e0 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_done global_id 4935 payload 274 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 1 -- 10.20.193.6:0/1845039553 >> v1:10.20.193.40:6789/0 conn(0x7f1c0815a5e0 legacy=0x7f1c0814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 1 -- 10.20.193.6:0/1845039553 >> v1:10.20.193.6:6789/0 conn(0x7f1c080c99a0 legacy=0x7f1c08159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 10 monclient: _finish_hunting 0 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 1 monclient: found mon.noname-a 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.749+0000 7f1c0e6fc640 1 -- 10.20.193.6:0/1845039553 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c080c8450 con 0x7f1c0806ff20 2026-02-22T23:26:47.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.751+0000 7f1c0e6fc640 1 --2- 10.20.193.6:0/1845039553 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c08070300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098d0 tx=0x7f1c0002eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c9e09a644d276 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:47.761 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 1 -- 10.20.193.6:0/1845039553 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0003c070 con 0x7f1c0806ff20 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: dump: 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:47.762 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: _finish_auth 0 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: _check_auth_tickets 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 1 -- 10.20.193.6:0/1845039553 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0002fba0 con 0x7f1c0806ff20 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 5 monclient: authenticate success, global_id 4935 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 1 -- 10.20.193.6:0/1845039553 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00037440 con 0x7f1c0806ff20 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 4 set_mon_vals no callback set 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:47.763 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c0d6fa640 10 monclient: dump: 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 10 monclient: shutdown 2026-02-22T23:26:47.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 1 -- 10.20.193.6:0/1845039553 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 msgr2=0x7f1c08070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 1 --2- 10.20.193.6:0/1845039553 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c08070300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c000098d0 tx=0x7f1c0002eed0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 1 -- 10.20.193.6:0/1845039553 shutdown_connections 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 1 --2- 10.20.193.6:0/1845039553 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c08070300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 1 -- 10.20.193.6:0/1845039553 >> 10.20.193.6:0/1845039553 conn(0x7f1c080cd780 msgr2=0x7f1c080cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 1 -- 10.20.193.6:0/1845039553 shutdown_connections 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.753+0000 7f1c10987640 1 -- 10.20.193.6:0/1845039553 wait complete. 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 10 monclient: build_initial_monmap 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 10 monclient: monmap: 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:47.765 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 1 librados: starting msgr at 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 1 librados: starting objecter 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 1 Processor -- start 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 1 -- start start 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 1 librados: setting wanted keys 2026-02-22T23:26:47.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 1 librados: calling monclient init 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.754+0000 7f1c10987640 10 monclient: init 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c0814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): picked mon.noname-a con 0x7f1c0806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c080c99a0 0x7f1c08156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): picked mon.noname-c con 0x7f1c080c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c0815a5e0 0x7f1c08156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): picked mon.noname-b con 0x7f1c0815a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08150520 con 0x7f1c0806ff20 2026-02-22T23:26:47.767 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080c8a10 con 0x7f1c0815a5e0 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081506a0 con 0x7f1c080c99a0 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c10987640 10 monclient(hunting): authenticate will time out at 1282.482186s 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c0814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c0814c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54800/0 (socket says 10.20.193.6:54800) 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 1 -- 10.20.193.6:0/850262750 learned_addr learned my addr 10.20.193.6:0/850262750 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 10 monclient(hunting): get_auth_request con 0x7f1c0806ff20 auth_method 0 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:47.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0eefd640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c0815a5e0 0x7f1c08156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0eefd640 10 monclient(hunting): get_auth_request con 0x7f1c0815a5e0 auth_method 0 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0eefd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0eefd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0eefd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.755+0000 7f1c0defb640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c080c99a0 0x7f1c08156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0defb640 10 monclient(hunting): get_auth_request con 0x7f1c080c99a0 auth_method 0 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0defb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0defb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:47.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0defb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0eefd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0eefd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0eefd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0defb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0defb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0defb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 10 monclient(hunting): handle_auth_done global_id 4941 payload 824 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 1 -- 10.20.193.6:0/850262750 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c080c99a0 msgr2=0x7f1c08156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c080c99a0 0x7f1c08156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 1 -- 10.20.193.6:0/850262750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c0815a5e0 msgr2=0x7f1c08156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c0815a5e0 0x7f1c08156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 10 monclient: _finish_hunting 0 2026-02-22T23:26:47.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 1 monclient: found mon.noname-a 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.756+0000 7f1c0e6fc640 1 -- 10.20.193.6:0/850262750 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0814d420 con 0x7f1c0806ff20 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.757+0000 7f1c0e6fc640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c0814c7d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009130 tx=0x7f1c000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.757+0000 7f1bf2ffd640 10 client.?.objecter ms_handle_connect 0x7f1c0806ff20 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.757+0000 7f1bf2ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 1 -- 10.20.193.6:0/850262750 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0003c040 con 0x7f1c0806ff20 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: dump: 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:47.771 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: _finish_auth 0 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 1 -- 10.20.193.6:0/850262750 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c00004230 con 0x7f1c0806ff20 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:47.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 1 -- 10.20.193.6:0/850262750 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004620 con 0x7f1c0806ff20 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 5 monclient: authenticate success, global_id 4941 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c0cef9640 4 set_mon_vals no callback set 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 monclient: _renew_subs 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c080bbbd0 con 0x7f1c0806ff20 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1bf2ffd640 10 monclient: dump: 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:47.773 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 client.4941.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 monclient: _renew_subs 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c08155690 con 0x7f1c0806ff20 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 1 librados: init done 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:47.774 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:47.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:47.775 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.759+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c08070300 con 0x7f1c0806ff20 2026-02-22T23:26:47.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.771+0000 7f1bf2ffd640 1 -- 10.20.193.6:0/850262750 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1c00003710 con 0x7f1c0806ff20 2026-02-22T23:26:47.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.771+0000 7f1bf2ffd640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1bd8078e50 0x7f1bd807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:47.776 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.771+0000 7f1c0defb640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1bd8078e50 0x7f1bd807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.771+0000 7f1c0defb640 10 monclient: get_auth_request con 0x7f1bd8078e50 auth_method 0 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.772+0000 7f1c0defb640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1bd8078e50 0x7f1bd807b310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1c081599f0 tx=0x7f1bf4007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.772+0000 7f1bf2ffd640 10 client.4941.objecter ms_handle_connect 0x7f1bd8078e50 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.772+0000 7f1bf2ffd640 1 -- 10.20.193.6:0/850262750 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f1c0002fa60 con 0x7f1c0806ff20 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.772+0000 7f1bf2ffd640 10 client.4941.objecter ms_dispatch2 0x7f1c08000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.772+0000 7f1bf2ffd640 3 client.4941.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.772+0000 7f1bf2ffd640 3 client.4941.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:47.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.772+0000 7f1bf2ffd640 20 client.4941.objecter dump_active .. 0 homeless 2026-02-22T23:26:47.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.871+0000 7f1bf2ffd640 1 -- 10.20.193.6:0/850262750 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1c00050070 con 0x7f1c0806ff20 2026-02-22T23:26:47.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.871+0000 7f1bf2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:47.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.871+0000 7f1bf2ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:47.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.971+0000 7f1c10987640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:47.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.971+0000 7f1c10987640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:47.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.971+0000 7f1c10987640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:47.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.971+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1c080c7330 con 0x7f1c0806ff20 2026-02-22T23:26:47.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.973+0000 7f1bf2ffd640 1 -- 10.20.193.6:0/850262750 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1c00046020 con 0x7f1c0806ff20 2026-02-22T23:26:47.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.973+0000 7f1bf2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:47.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.973+0000 7f1bf2ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:47.975 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:47.975 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:47.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 10 librados: watch_flush enter 2026-02-22T23:26:47.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 10 librados: watch_flush exit 2026-02-22T23:26:47.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 20 client.4941.objecter shutdown clearing up homeless session... 2026-02-22T23:26:47.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 10 client.4941.objecter successfully canceled tick 2026-02-22T23:26:47.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1bd8078e50 msgr2=0x7f1bd807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:47.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1bd8078e50 0x7f1bd807b310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1c081599f0 tx=0x7f1bf4007ef0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 10 monclient: shutdown 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 msgr2=0x7f1c0814c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c0814c7d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009130 tx=0x7f1c000047c0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 shutdown_connections 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1bd8078e50 0x7f1bd807b310 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1c0815a5e0 0x7f1c08156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1c080c99a0 0x7f1c08156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 --2- 10.20.193.6:0/850262750 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1c0806ff20 0x7f1c0814c7d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 >> 10.20.193.6:0/850262750 conn(0x7f1c080cd780 msgr2=0x7f1c0814e600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 shutdown_connections 2026-02-22T23:26:47.978 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.974+0000 7f1c10987640 1 -- 10.20.193.6:0/850262750 wait complete. 2026-02-22T23:26:47.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:47.975+0000 7f1c10987640 1 librados: shutdown 2026-02-22T23:26:50.985 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:51.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient: get_monmap_and_config 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient: build_initial_monmap 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient: monmap: 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:51.054664+0000 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:51.054664+0000 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:51.056 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 1 Processor -- start 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 1 -- start start 2026-02-22T23:26:51.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient: init 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient: _add_conns ranks=[4,5,1,2,0,3] 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f822406ff20 0x7f8224070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient(hunting): picked mon.noname-b con 0x7f822406ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f82240c99a0 0x7f8224159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient(hunting): picked mon.noname-c con 0x7f82240c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f822415a410 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:51.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822415a860 con 0x7f822415a410 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f8224159a40 con 0x7f822406ff20 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f82240c7e90 con 0x7f82240c99a0 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.054+0000 7f822b824640 10 monclient(hunting): authenticate will time out at 1285.781992s 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8229599640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f822406ff20 0x7f8224070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8228d98640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f82240c99a0 0x7f8224159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8229599640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f822406ff20 0x7f8224070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:48816/0 (socket says 10.20.193.6:48816) 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8229599640 1 -- 10.20.193.6:0/878948355 learned_addr learned my addr 10.20.193.6:0/878948355 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8229599640 10 monclient(hunting): get_auth_request con 0x7f822406ff20 auth_method 0 2026-02-22T23:26:51.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8229599640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8229599640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8229599640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8228d98640 10 monclient(hunting): get_auth_request con 0x7f82240c99a0 auth_method 0 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8228d98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8228d98640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.055+0000 7f8228d98640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.056+0000 7f8229599640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.056+0000 7f8229599640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.056+0000 7f8229599640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.056+0000 7f8228d98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.056+0000 7f8228d98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:51.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.056+0000 7f8228d98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 10 monclient(hunting): handle_auth_done global_id 5027 payload 274 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 1 -- 10.20.193.6:0/878948355 >> v2:10.20.193.40:3300/0 conn(0x7f822406ff20 msgr2=0x7f8224070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 1 --2- 10.20.193.6:0/878948355 >> v2:10.20.193.40:3300/0 conn(0x7f822406ff20 0x7f8224070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 1 -- 10.20.193.6:0/878948355 >> v1:10.20.193.40:6789/0 conn(0x7f822415a410 legacy=0x7f822414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 10 monclient: _finish_hunting 0 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 1 monclient: found mon.noname-c 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.057+0000 7f8228d98640 1 -- 10.20.193.6:0/878948355 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82240c8170 con 0x7f82240c99a0 2026-02-22T23:26:51.063 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.058+0000 7f8228d98640 1 --2- 10.20.193.6:0/878948355 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f8224159ed0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f82180098a0 tx=0x7f821802ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd75e7c699e7b95 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:51.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 1 -- 10.20.193.6:0/878948355 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821803c070 con 0x7f82240c99a0 2026-02-22T23:26:51.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: dump: 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:51.070 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: _finish_auth 0 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: _check_auth_tickets 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 1 -- 10.20.193.6:0/878948355 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821802fa40 con 0x7f82240c99a0 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f822b824640 5 monclient: authenticate success, global_id 5027 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 1 -- 10.20.193.6:0/878948355 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821802fe10 con 0x7f82240c99a0 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f822b824640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f822b824640 4 set_mon_vals no callback set 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:51.071 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.060+0000 7f8213fff640 10 monclient: dump: 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 10 monclient: shutdown 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:51.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 -- 10.20.193.6:0/878948355 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 msgr2=0x7f8224159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 --2- 10.20.193.6:0/878948355 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f8224159ed0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f82180098a0 tx=0x7f821802ee10 comp rx=0 tx=0).stop 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 -- 10.20.193.6:0/878948355 shutdown_connections 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 --2- 10.20.193.6:0/878948355 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f8224159ed0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 --2- 10.20.193.6:0/878948355 >> v2:10.20.193.40:3300/0 conn(0x7f822406ff20 0x7f8224070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 -- 10.20.193.6:0/878948355 >> 10.20.193.6:0/878948355 conn(0x7f82240cd780 msgr2=0x7f82240cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 -- 10.20.193.6:0/878948355 shutdown_connections 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 1 -- 10.20.193.6:0/878948355 wait complete. 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 10 monclient: build_initial_monmap 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.061+0000 7f822b824640 10 monclient: monmap: 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:51.073 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 librados: starting msgr at 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 librados: starting objecter 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 Processor -- start 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 -- start start 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 librados: setting wanted keys 2026-02-22T23:26:51.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 librados: calling monclient init 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient: init 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f822406ff20 0x7f82241f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): picked mon.noname-b con 0x7f822406ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f82240cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): picked mon.noname-c con 0x7f82240c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f822415a410 0x7f8224154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): picked mon.noname-a con 0x7f822415a410 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f82240c8a10 con 0x7f822415a410 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:51.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82240c8730 con 0x7f822406ff20 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8224150520 con 0x7f82240c99a0 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f822b824640 10 monclient(hunting): authenticate will time out at 1285.789700s 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.062+0000 7f8229d9a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f822415a410 0x7f8224154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f82240cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f822406ff20 0x7f82241f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f822415a410 0x7f8224154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41862/0 (socket says 10.20.193.6:41862) 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 1 -- 10.20.193.6:0/942611132 learned_addr learned my addr 10.20.193.6:0/942611132 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 10 monclient(hunting): get_auth_request con 0x7f822415a410 auth_method 0 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:51.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 10 monclient(hunting): get_auth_request con 0x7f822406ff20 auth_method 0 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 10 monclient(hunting): get_auth_request con 0x7f82240c99a0 auth_method 0 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229d9a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:51.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8228d98640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.063+0000 7f8229599640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 10 monclient(hunting): handle_auth_done global_id 5030 payload 824 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 1 -- 10.20.193.6:0/942611132 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f822406ff20 msgr2=0x7f82241f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f822406ff20 0x7f82241f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 1 -- 10.20.193.6:0/942611132 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f822415a410 msgr2=0x7f8224154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f822415a410 0x7f8224154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 10 monclient: _finish_hunting 0 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 1 monclient: found mon.noname-c 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:51.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.064+0000 7f8228d98640 1 -- 10.20.193.6:0/942611132 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8224154f90 con 0x7f82240c99a0 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.065+0000 7f8228d98640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f82240cbdd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8218036000 tx=0x7f82180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.065+0000 7f8211ffb640 10 client.?.objecter ms_handle_connect 0x7f82240c99a0 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.065+0000 7f8211ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 1 -- 10.20.193.6:0/942611132 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218046020 con 0x7f82240c99a0 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: dump: 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.079 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: _finish_auth 0 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 1 -- 10.20.193.6:0/942611132 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8218042650 con 0x7f82240c99a0 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 5 monclient: authenticate success, global_id 5030 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 10 monclient: _renew_subs 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:51.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8224155270 con 0x7f82240c99a0 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 1 -- 10.20.193.6:0/942611132 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821803c040 con 0x7f82240c99a0 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f82137fe640 4 set_mon_vals no callback set 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f8211ffb640 10 monclient: dump: 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:51.081 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 10 client.5030.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 10 monclient: _renew_subs 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8224155830 con 0x7f82240c99a0 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.067+0000 7f822b824640 1 librados: init done 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.068+0000 7f822b824640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.068+0000 7f822b824640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.068+0000 7f822b824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:51.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.068+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82241508b0 con 0x7f82240c99a0 2026-02-22T23:26:51.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.079+0000 7f8211ffb640 1 -- 10.20.193.6:0/942611132 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f821804b440 con 0x7f82240c99a0 2026-02-22T23:26:51.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.080+0000 7f8211ffb640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81f4078e50 0x7f81f407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:51.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.080+0000 7f8229599640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81f4078e50 0x7f81f407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:51.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.080+0000 7f8229599640 10 monclient: get_auth_request con 0x7f81f4078e50 auth_method 0 2026-02-22T23:26:51.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.081+0000 7f8229599640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81f4078e50 0x7f81f407b310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8214006dc0 tx=0x7f8214009340 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:51.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.081+0000 7f8211ffb640 10 client.5030.objecter ms_handle_connect 0x7f81f4078e50 2026-02-22T23:26:51.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.174+0000 7f8211ffb640 1 -- 10.20.193.6:0/942611132 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f821802f9a0 con 0x7f82240c99a0 2026-02-22T23:26:51.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.174+0000 7f8211ffb640 10 client.5030.objecter ms_dispatch2 0x7f8224000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:51.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.174+0000 7f8211ffb640 3 client.5030.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:51.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.174+0000 7f8211ffb640 3 client.5030.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:51.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.174+0000 7f8211ffb640 20 client.5030.objecter dump_active .. 0 homeless 2026-02-22T23:26:51.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.180+0000 7f8211ffb640 1 -- 10.20.193.6:0/942611132 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f82180866a0 con 0x7f82240c99a0 2026-02-22T23:26:51.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.180+0000 7f8211ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:51.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.180+0000 7f8211ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:51.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.282+0000 7f822b824640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:51.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.282+0000 7f822b824640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:51.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.282+0000 7f822b824640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:26:51.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.282+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f82241573f0 con 0x7f82240c99a0 2026-02-22T23:26:51.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.285+0000 7f8211ffb640 1 -- 10.20.193.6:0/942611132 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f821808a9a0 con 0x7f82240c99a0 2026-02-22T23:26:51.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.285+0000 7f8211ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:51.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.285+0000 7f8211ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:51.287 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:51.287 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:51.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 10 librados: watch_flush enter 2026-02-22T23:26:51.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 10 librados: watch_flush exit 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 20 client.5030.objecter shutdown clearing up homeless session... 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 10 client.5030.objecter successfully canceled tick 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81f4078e50 msgr2=0x7f81f407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81f4078e50 0x7f81f407b310 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8214006dc0 tx=0x7f8214009340 comp rx=0 tx=0).stop 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 10 monclient: shutdown 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 msgr2=0x7f82240cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f82240cbdd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8218036000 tx=0x7f82180047c0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 shutdown_connections 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81f4078e50 0x7f81f407b310 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f822415a410 0x7f8224154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82240c99a0 0x7f82240cbdd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 --2- 10.20.193.6:0/942611132 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f822406ff20 0x7f82241f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:51.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 >> 10.20.193.6:0/942611132 conn(0x7f82240cd780 msgr2=0x7f82240c4690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:51.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 shutdown_connections 2026-02-22T23:26:51.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 -- 10.20.193.6:0/942611132 wait complete. 2026-02-22T23:26:51.290 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:51.286+0000 7f822b824640 1 librados: shutdown 2026-02-22T23:26:53.249 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:37.498 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.277 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:37.526 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.277 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:37.524 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.279 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:37.525 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.287 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:37.537 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.287 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:37.534 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.308 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:37.519 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.308 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:37.520 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:53.317 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:37.528 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:26:54.296 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:54.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.363+0000 7fc20963c640 10 monclient: get_monmap_and_config 2026-02-22T23:26:54.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.363+0000 7fc20963c640 10 monclient: build_initial_monmap 2026-02-22T23:26:54.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.363+0000 7fc20963c640 10 monclient: monmap: 2026-02-22T23:26:54.366 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:54.366 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:54.366 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:54.363767+0000 2026-02-22T23:26:54.366 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:54.363767+0000 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.363+0000 7fc20963c640 1 Processor -- start 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.363+0000 7fc20963c640 1 -- start start 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.363+0000 7fc20963c640 10 monclient: init 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient: _add_conns ranks=[5,4,3,0,2,1] 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc20406ff20 0x7fc204070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): picked mon.noname-c con 0x7fc20406ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc2040c99a0 0x7fc204159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): picked mon.noname-b con 0x7fc2040c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc20415a410 0x7fc20414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): picked mon.noname-a con 0x7fc20415a410 addr v2:10.20.193.6:3300/0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc204159a40 con 0x7fc20415a410 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc20415a860 con 0x7fc2040c99a0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc2040c7e90 con 0x7fc20406ff20 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc20963c640 10 monclient(hunting): authenticate will time out at 1289.091111s 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2037fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc20415a410 0x7fc20414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2027fc640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc2040c99a0 0x7fc204159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc202ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc20406ff20 0x7fc204070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2037fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc20415a410 0x7fc20414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41866/0 (socket says 10.20.193.6:41866) 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2037fe640 1 -- 10.20.193.6:0/53882106 learned_addr learned my addr 10.20.193.6:0/53882106 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2037fe640 10 monclient(hunting): get_auth_request con 0x7fc20415a410 auth_method 0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2037fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc202ffd640 10 monclient(hunting): get_auth_request con 0x7fc20406ff20 auth_method 0 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc202ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc202ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc202ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:54.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2027fc640 10 monclient(hunting): get_auth_request con 0x7fc2040c99a0 auth_method 0 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2027fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.364+0000 7fc2027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc2037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc2037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc2037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc202ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc202ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc202ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc2027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc2027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.365+0000 7fc2027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 10 monclient(hunting): handle_auth_done global_id 4947 payload 274 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 1 -- 10.20.193.6:0/53882106 >> v2:10.20.193.80:3300/0 conn(0x7fc20406ff20 msgr2=0x7fc204070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 1 --2- 10.20.193.6:0/53882106 >> v2:10.20.193.80:3300/0 conn(0x7fc20406ff20 0x7fc204070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 1 -- 10.20.193.6:0/53882106 >> v2:10.20.193.40:3300/0 conn(0x7fc2040c99a0 msgr2=0x7fc204159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 1 --2- 10.20.193.6:0/53882106 >> v2:10.20.193.40:3300/0 conn(0x7fc2040c99a0 0x7fc204159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 10 monclient: _finish_hunting 0 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 1 monclient: found mon.noname-a 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.366+0000 7fc2037fe640 1 -- 10.20.193.6:0/53882106 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040c8170 con 0x7fc20415a410 2026-02-22T23:26:54.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.367+0000 7fc2037fe640 1 --2- 10.20.193.6:0/53882106 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc20414fe10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc2040709c0 tx=0x7fc1f402f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88af5d0c3f506137 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 1 -- 10.20.193.6:0/53882106 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f402fce0 con 0x7fc20415a410 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: dump: 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: _finish_auth 0 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 1 -- 10.20.193.6:0/53882106 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f40373f0 con 0x7fc20415a410 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 5 monclient: authenticate success, global_id 4947 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 1 -- 10.20.193.6:0/53882106 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40377c0 con 0x7fc20415a410 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc201ffb640 10 monclient: dump: 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.370 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 4 set_mon_vals no callback set 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 10 monclient: shutdown 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 -- 10.20.193.6:0/53882106 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 msgr2=0x7fc20414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 --2- 10.20.193.6:0/53882106 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc20414fe10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc2040709c0 tx=0x7fc1f402f270 comp rx=0 tx=0).stop 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 -- 10.20.193.6:0/53882106 shutdown_connections 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 --2- 10.20.193.6:0/53882106 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc20414fe10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 --2- 10.20.193.6:0/53882106 >> v2:10.20.193.40:3300/0 conn(0x7fc2040c99a0 0x7fc204159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 --2- 10.20.193.6:0/53882106 >> v2:10.20.193.80:3300/0 conn(0x7fc20406ff20 0x7fc204070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 -- 10.20.193.6:0/53882106 >> 10.20.193.6:0/53882106 conn(0x7fc2040cd780 msgr2=0x7fc2040cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 -- 10.20.193.6:0/53882106 shutdown_connections 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.369+0000 7fc20963c640 1 -- 10.20.193.6:0/53882106 wait complete. 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 10 monclient: build_initial_monmap 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 10 monclient: monmap: 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:54.371 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 1 librados: starting msgr at 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 1 librados: starting objecter 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 1 Processor -- start 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 1 -- start start 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 1 librados: setting wanted keys 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 1 librados: calling monclient init 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 10 monclient: init 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.370+0000 7fc20963c640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:26:54.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20406ff20 0x7fc2041f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): picked mon.noname-c con 0x7fc20406ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2040c99a0 0x7fc2040c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): picked mon.noname-b con 0x7fc2040c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc2040c0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): picked mon.noname-a con 0x7fc20415a410 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc2040c8a10 con 0x7fc20415a410 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2040c8730 con 0x7fc2040c99a0 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc204150350 con 0x7fc20406ff20 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc20963c640 10 monclient(hunting): authenticate will time out at 1289.098048s 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc2040c0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc2040c0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41878/0 (socket says 10.20.193.6:41878) 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 1 -- 10.20.193.6:0/2811209879 learned_addr learned my addr 10.20.193.6:0/2811209879 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 10 monclient(hunting): get_auth_request con 0x7fc20415a410 auth_method 0 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20406ff20 0x7fc2041f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2027fc640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2040c99a0 0x7fc2040c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 10 monclient(hunting): get_auth_request con 0x7fc20406ff20 auth_method 0 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2037fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2027fc640 10 monclient(hunting): get_auth_request con 0x7fc2040c99a0 auth_method 0 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2027fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2027fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc2027fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:54.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.371+0000 7fc202ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.372+0000 7fc2027fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.372+0000 7fc2027fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.372+0000 7fc2027fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 10 monclient(hunting): handle_auth_done global_id 4950 payload 824 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 1 -- 10.20.193.6:0/2811209879 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20406ff20 msgr2=0x7fc2041f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20406ff20 0x7fc2041f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 1 -- 10.20.193.6:0/2811209879 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2040c99a0 msgr2=0x7fc2040c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2040c99a0 0x7fc2040c3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 10 monclient: _finish_hunting 0 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 1 monclient: found mon.noname-a 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:54.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.373+0000 7fc2037fe640 1 -- 10.20.193.6:0/2811209879 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2040c22c0 con 0x7fc20415a410 2026-02-22T23:26:54.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.374+0000 7fc2037fe640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc2040c0780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4039040 tx=0x7fc1f4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:54.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.374+0000 7fc1e37fe640 10 client.?.objecter ms_handle_connect 0x7fc20415a410 2026-02-22T23:26:54.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.374+0000 7fc1e37fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 1 -- 10.20.193.6:0/2811209879 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4046020 con 0x7fc20415a410 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: dump: 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:54.378 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: _finish_auth 0 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: _check_auth_tickets 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 5 monclient: authenticate success, global_id 4950 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 monclient: _renew_subs 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:54.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2040c25a0 con 0x7fc20415a410 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 client.4950.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 monclient: _renew_subs 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2040c2b60 con 0x7fc20415a410 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 1 librados: init done 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2041506e0 con 0x7fc20415a410 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 1 -- 10.20.193.6:0/2811209879 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f40373f0 con 0x7fc20415a410 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 1 -- 10.20.193.6:0/2811209879 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4009510 con 0x7fc20415a410 2026-02-22T23:26:54.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc1e37fe640 10 monclient: dump: 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:54.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.375+0000 7fc2017fa640 4 set_mon_vals no callback set 2026-02-22T23:26:54.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.479+0000 7fc1e37fe640 1 -- 10.20.193.6:0/2811209879 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc1f4041650 con 0x7fc20415a410 2026-02-22T23:26:54.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.479+0000 7fc1e37fe640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1d0078e00 0x7fc1d007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:54.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.480+0000 7fc202ffd640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1d0078e00 0x7fc1d007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:54.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.480+0000 7fc202ffd640 10 monclient: get_auth_request con 0x7fc1d0078e00 auth_method 0 2026-02-22T23:26:54.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.480+0000 7fc202ffd640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1d0078e00 0x7fc1d007b2c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009c40 tx=0x7fc1f8009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:54.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.480+0000 7fc1e37fe640 10 client.4950.objecter ms_handle_connect 0x7fc1d0078e00 2026-02-22T23:26:54.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 1 -- 10.20.193.6:0/2811209879 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc1f40bcfa0 con 0x7fc20415a410 2026-02-22T23:26:54.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 10 client.4950.objecter ms_dispatch2 0x7fc204000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:54.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 3 client.4950.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:54.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 3 client.4950.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:54.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 20 client.4950.objecter dump_active .. 0 homeless 2026-02-22T23:26:54.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 1 -- 10.20.193.6:0/2811209879 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc1f4050030 con 0x7fc20415a410 2026-02-22T23:26:54.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:54.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.487+0000 7fc1e37fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:54.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.590+0000 7fc20963c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:54.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.590+0000 7fc20963c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:54.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.590+0000 7fc20963c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:54.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.590+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc204157a90 con 0x7fc20415a410 2026-02-22T23:26:54.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.592+0000 7fc1e37fe640 1 -- 10.20.193.6:0/2811209879 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc1f4087770 con 0x7fc20415a410 2026-02-22T23:26:54.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.592+0000 7fc1e37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:54.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.592+0000 7fc1e37fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:54.594 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:54.594 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 10 librados: watch_flush enter 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 10 librados: watch_flush exit 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 20 client.4950.objecter shutdown clearing up homeless session... 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 10 client.4950.objecter successfully canceled tick 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1d0078e00 msgr2=0x7fc1d007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1d0078e00 0x7fc1d007b2c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009c40 tx=0x7fc1f8009290 comp rx=0 tx=0).stop 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 10 monclient: shutdown 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 msgr2=0x7fc2040c0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc2040c0780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4039040 tx=0x7fc1f4004b40 comp rx=0 tx=0).stop 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 shutdown_connections 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1d0078e00 0x7fc1d007b2c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20415a410 0x7fc2040c0780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2040c99a0 0x7fc2040c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 --2- 10.20.193.6:0/2811209879 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20406ff20 0x7fc2041f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 >> 10.20.193.6:0/2811209879 conn(0x7fc2040cd780 msgr2=0x7fc20414fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 shutdown_connections 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 -- 10.20.193.6:0/2811209879 wait complete. 2026-02-22T23:26:54.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:54.594+0000 7fc20963c640 1 librados: shutdown 2026-02-22T23:26:57.604 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:26:57.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.670+0000 7f81fa144640 10 monclient: get_monmap_and_config 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.670+0000 7f81fa144640 10 monclient: build_initial_monmap 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.670+0000 7f81fa144640 10 monclient: monmap: 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:26:57.671472+0000 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:26:57.671472+0000 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:26:57.673 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 1 Processor -- start 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 1 -- start start 2026-02-22T23:26:57.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient: init 2026-02-22T23:26:57.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient: _add_conns ranks=[0,1,4,2,5,3] 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f81f41527d0 addr v1:10.20.193.6:6789/0 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f81f4152bb0 addr v1:10.20.193.40:6789/0 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f81f40c6190 0x7f81f40c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient(hunting): picked mon.noname-b con 0x7f81f40c6190 addr v2:10.20.193.40:3300/0 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f406e300 con 0x7f81f41527d0 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81f406e5e0 con 0x7f81f4152bb0 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f81f406f1f0 con 0x7f81f40c6190 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:57.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.671+0000 7f81fa144640 10 monclient(hunting): authenticate will time out at 1292.398834s 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f37fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f81f41527d0 0x7f81f406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:41058/0 (socket says 10.20.193.6:41058) 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f37fe640 1 -- 10.20.193.6:0/4038992082 learned_addr learned my addr 10.20.193.6:0/4038992082 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 1 --2- 10.20.193.6:0/4038992082 >> v2:10.20.193.40:3300/0 conn(0x7f81f40c6190 0x7f81f40c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 10 monclient(hunting): get_auth_request con 0x7f81f40c6190 auth_method 0 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.672+0000 7f81f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f27fc640 1 -- 10.20.193.6:0/4038992082 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364663681 0 0) 0x7f81f406e300 con 0x7f81f41527d0 2026-02-22T23:26:57.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f27fc640 10 monclient(hunting): _negotiate 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f27fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f27fc640 10 monclient(hunting): my global_id is 4953 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f27fc640 1 -- 10.20.193.6:0/4038992082 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81d8003620 con 0x7f81f41527d0 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 10 monclient(hunting): handle_auth_done global_id 4915 payload 274 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 1 -- 10.20.193.6:0/4038992082 >> v1:10.20.193.40:6789/0 conn(0x7f81f4152bb0 legacy=0x7f81f40c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 1 -- 10.20.193.6:0/4038992082 >> v1:10.20.193.6:6789/0 conn(0x7f81f41527d0 legacy=0x7f81f406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 1 monclient: found mon.noname-b 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.673+0000 7f81f3fff640 1 -- 10.20.193.6:0/4038992082 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f406e8c0 con 0x7f81f40c6190 2026-02-22T23:26:57.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.675+0000 7f81f3fff640 1 --2- 10.20.193.6:0/4038992082 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 0x7f81f40c8e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f81f406f370 tx=0x7f81e802f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947e9a48358e1652 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:57.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 1 -- 10.20.193.6:0/4038992082 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e803c070 con 0x7f81f40c6190 2026-02-22T23:26:57.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:57.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:26:57.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: dump: 2026-02-22T23:26:57.686 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: _finish_auth 0 2026-02-22T23:26:57.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: _check_auth_tickets 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 1 -- 10.20.193.6:0/4038992082 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e8037440 con 0x7f81f40c6190 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 1 -- 10.20.193.6:0/4038992082 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8037810 con 0x7f81f40c6190 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 5 monclient: authenticate success, global_id 4915 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81f27fc640 10 monclient: dump: 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.688 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 10 monclient: get_monmap_and_config success 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 4 set_mon_vals no callback set 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 10 monclient: shutdown 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 1 -- 10.20.193.6:0/4038992082 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 msgr2=0x7f81f40c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:57.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 1 --2- 10.20.193.6:0/4038992082 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 0x7f81f40c8e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f81f406f370 tx=0x7f81e802f1e0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 1 -- 10.20.193.6:0/4038992082 shutdown_connections 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 1 --2- 10.20.193.6:0/4038992082 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 0x7f81f40c8e60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 1 -- 10.20.193.6:0/4038992082 >> 10.20.193.6:0/4038992082 conn(0x7f81f40cd780 msgr2=0x7f81f40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 1 -- 10.20.193.6:0/4038992082 shutdown_connections 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.677+0000 7f81fa144640 1 -- 10.20.193.6:0/4038992082 wait complete. 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient: build_initial_monmap 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient: monmap: 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:26:57.690 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 librados: starting msgr at 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 librados: starting objecter 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 Processor -- start 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 -- start start 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 librados: setting wanted keys 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 librados: calling monclient init 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient: init 2026-02-22T23:26:57.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient: _reopen_session rank -1 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 0x7f81f41434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): picked mon.noname-b con 0x7f81f40c6190 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f81f41527d0 0x7f81f414efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): picked mon.noname-c con 0x7f81f41527d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f81f4152bb0 0x7f81f4145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): picked mon.noname-a con 0x7f81f4152bb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f81f40c93a0 con 0x7f81f4152bb0 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f81f406ee80 con 0x7f81f40c6190 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): start opening mon connection 2026-02-22T23:26:57.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f81f40c9520 con 0x7f81f41527d0 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): _renew_subs 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.678+0000 7f81fa144640 10 monclient(hunting): authenticate will time out at 1292.405813s 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f81f4152bb0 0x7f81f4145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 0x7f81f41434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f81f4152bb0 0x7f81f4145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41886/0 (socket says 10.20.193.6:41886) 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 1 -- 10.20.193.6:0/276420310 learned_addr learned my addr 10.20.193.6:0/276420310 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 10 monclient(hunting): get_auth_request con 0x7f81f4152bb0 auth_method 0 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 10 monclient(hunting): get_auth_request con 0x7f81f40c6190 auth_method 0 2026-02-22T23:26:57.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f2ffd640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f81f41527d0 0x7f81f414efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f2ffd640 10 monclient(hunting): get_auth_request con 0x7f81f41527d0 auth_method 0 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:26:57.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.679+0000 7f81f2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.680+0000 7f81f2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.680+0000 7f81f2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.680+0000 7f81f2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 10 monclient(hunting): handle_auth_done global_id 4956 payload 824 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 1 -- 10.20.193.6:0/276420310 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f81f41527d0 msgr2=0x7f81f414efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f81f41527d0 0x7f81f414efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 1 -- 10.20.193.6:0/276420310 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 msgr2=0x7f81f41434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 0x7f81f41434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 10 monclient: _finish_hunting 0 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 1 monclient: found mon.noname-a 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:26:57.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.681+0000 7f81f3fff640 1 -- 10.20.193.6:0/276420310 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f414db30 con 0x7f81f4152bb0 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.682+0000 7f81f3fff640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f81f4152bb0 0x7f81f4145d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009e90 tx=0x7f81e8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.682+0000 7f81cffff640 10 client.?.objecter ms_handle_connect 0x7f81f4152bb0 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.682+0000 7f81cffff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 1 -- 10.20.193.6:0/276420310 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8047020 con 0x7f81f4152bb0 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: dump: 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.696 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: _finish_auth 0 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: _check_auth_tickets 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 1 -- 10.20.193.6:0/276420310 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81e8042670 con 0x7f81f4152bb0 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 1 -- 10.20.193.6:0/276420310 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e803c040 con 0x7f81f4152bb0 2026-02-22T23:26:57.697 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81cffff640 10 monclient: dump: 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:26:57.698 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81f1ffb640 4 set_mon_vals no callback set 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 5 monclient: authenticate success, global_id 4956 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 10 monclient: _renew_subs 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f4073e60 con 0x7f81f4152bb0 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 10 client.4956.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 10 monclient: _renew_subs 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f4074420 con 0x7f81f4152bb0 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.683+0000 7f81fa144640 1 librados: init done 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.684+0000 7f81fa144640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.684+0000 7f81fa144640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.684+0000 7f81fa144640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:57.699 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.684+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f40c8e60 con 0x7f81f4152bb0 2026-02-22T23:26:57.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.696+0000 7f81cffff640 1 -- 10.20.193.6:0/276420310 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f81e8042a60 con 0x7f81f4152bb0 2026-02-22T23:26:57.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.696+0000 7f81cffff640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81c4078e50 0x7f81c407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:26:57.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.697+0000 7f81f37fe640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81c4078e50 0x7f81c407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:26:57.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.697+0000 7f81f37fe640 10 monclient: get_auth_request con 0x7f81c4078e50 auth_method 0 2026-02-22T23:26:57.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.697+0000 7f81f37fe640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81c4078e50 0x7f81c407b310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81e0007b60 tx=0x7f81e0007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:26:57.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.697+0000 7f81cffff640 10 client.4956.objecter ms_handle_connect 0x7f81c4078e50 2026-02-22T23:26:57.795 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.793+0000 7f81cffff640 1 -- 10.20.193.6:0/276420310 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f81e8042d80 con 0x7f81f4152bb0 2026-02-22T23:26:57.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.794+0000 7f81cffff640 10 client.4956.objecter ms_dispatch2 0x7f81f4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:26:57.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.794+0000 7f81cffff640 3 client.4956.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:26:57.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.794+0000 7f81cffff640 3 client.4956.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:26:57.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.794+0000 7f81cffff640 20 client.4956.objecter dump_active .. 0 homeless 2026-02-22T23:26:57.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.794+0000 7f81cffff640 1 -- 10.20.193.6:0/276420310 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f81e8051030 con 0x7f81f4152bb0 2026-02-22T23:26:57.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.794+0000 7f81cffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:26:57.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.794+0000 7f81cffff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:26:57.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.895+0000 7f81fa144640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:26:57.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.895+0000 7f81fa144640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:57.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.895+0000 7f81fa144640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:26:57.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.895+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f81f41562f0 con 0x7f81f4152bb0 2026-02-22T23:26:57.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.909+0000 7f81cffff640 1 -- 10.20.193.6:0/276420310 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f81e80880f0 con 0x7f81f4152bb0 2026-02-22T23:26:57.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.909+0000 7f81cffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:26:57.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.909+0000 7f81cffff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:26:57.911 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:26:57.911 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:26:57.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 10 librados: watch_flush enter 2026-02-22T23:26:57.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 10 librados: watch_flush exit 2026-02-22T23:26:57.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 20 client.4956.objecter shutdown clearing up homeless session... 2026-02-22T23:26:57.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 10 client.4956.objecter successfully canceled tick 2026-02-22T23:26:57.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81c4078e50 msgr2=0x7f81c407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:57.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81c4078e50 0x7f81c407b310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f81e0007b60 tx=0x7f81e0007ab0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.913 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 10 monclient: shutdown 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f81f4152bb0 msgr2=0x7f81f4145d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f81f4152bb0 0x7f81f4145d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009e90 tx=0x7f81e8004b40 comp rx=0 tx=0).stop 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 shutdown_connections 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81c4078e50 0x7f81c407b310 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f81f4152bb0 0x7f81f4145d80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f81f41527d0 0x7f81f414efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 --2- 10.20.193.6:0/276420310 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f81f40c6190 0x7f81f41434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 >> 10.20.193.6:0/276420310 conn(0x7f81f40cd780 msgr2=0x7f81f4147300 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 shutdown_connections 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.910+0000 7f81fa144640 1 -- 10.20.193.6:0/276420310 wait complete. 2026-02-22T23:26:57.914 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:26:57.911+0000 7f81fa144640 1 librados: shutdown 2026-02-22T23:27:00.922 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.987+0000 7f3d63a16640 10 monclient: get_monmap_and_config 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.987+0000 7f3d63a16640 10 monclient: build_initial_monmap 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.987+0000 7f3d63a16640 10 monclient: monmap: 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:00.988723+0000 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:00.988723+0000 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:00.989 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.987+0000 7f3d63a16640 1 Processor -- start 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.987+0000 7f3d63a16640 1 -- start start 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.987+0000 7f3d63a16640 10 monclient: init 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient: _add_conns ranks=[5,2,0,1,4,3] 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f3d5c06ff20 0x7f3d5c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient(hunting): picked mon.noname-c con 0x7f3d5c06ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3d5c0c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3d5c15a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5c0c7e90 con 0x7f3d5c15a5e0 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5c0c8170 con 0x7f3d5c0c99a0 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3d5c159a40 con 0x7f3d5c06ff20 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d63a16640 10 monclient(hunting): authenticate will time out at 1295.716107s 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d61f8c640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f3d5c15a5e0 0x7f3d5c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:56488/0 (socket says 10.20.193.6:56488) 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d61f8c640 1 -- 10.20.193.6:0/1446526682 learned_addr learned my addr 10.20.193.6:0/1446526682 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:00.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.988+0000 7f3d6178b640 1 --2- 10.20.193.6:0/1446526682 >> v2:10.20.193.80:3300/0 conn(0x7f3d5c06ff20 0x7f3d5c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d6178b640 10 monclient(hunting): get_auth_request con 0x7f3d5c06ff20 auth_method 0 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d6178b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d6178b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d6178b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d6178b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d6178b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d6178b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d4bfff640 1 -- 10.20.193.6:0/1446526682 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4236889491 0 0) 0x7f3d5c0c7e90 con 0x7f3d5c15a5e0 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d4bfff640 10 monclient(hunting): _negotiate 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d4bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d4bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d4bfff640 10 monclient(hunting): my global_id is 4959 2026-02-22T23:27:00.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.989+0000 7f3d4bfff640 1 -- 10.20.193.6:0/1446526682 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d44003620 con 0x7f3d5c15a5e0 2026-02-22T23:27:00.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 10 monclient(hunting): handle_auth_done global_id 5042 payload 274 2026-02-22T23:27:00.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 1 -- 10.20.193.6:0/1446526682 >> v1:10.20.193.80:6789/0 conn(0x7f3d5c0c99a0 legacy=0x7f3d5c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:00.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 1 -- 10.20.193.6:0/1446526682 >> v1:10.20.193.6:6789/0 conn(0x7f3d5c15a5e0 legacy=0x7f3d5c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:00.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 10 monclient: _finish_hunting 0 2026-02-22T23:27:00.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 1 monclient: found mon.noname-c 2026-02-22T23:27:00.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:00.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:00.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.990+0000 7f3d6178b640 1 -- 10.20.193.6:0/1446526682 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c0c8450 con 0x7f3d5c06ff20 2026-02-22T23:27:00.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.992+0000 7f3d6178b640 1 --2- 10.20.193.6:0/1446526682 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c070300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0098d0 tx=0x7f3d4c02eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ec8cd5fa74fb8e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 1 -- 10.20.193.6:0/1446526682 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c03c070 con 0x7f3d5c06ff20 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: dump: 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: _finish_auth 0 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 1 -- 10.20.193.6:0/1446526682 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c02fba0 con 0x7f3d5c06ff20 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 1 -- 10.20.193.6:0/1446526682 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c037440 con 0x7f3d5c06ff20 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 5 monclient: authenticate success, global_id 5042 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d4bfff640 10 monclient: dump: 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:00.996 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 4 set_mon_vals no callback set 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 10 monclient: shutdown 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 1 -- 10.20.193.6:0/1446526682 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c06ff20 msgr2=0x7f3d5c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:00.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 1 --2- 10.20.193.6:0/1446526682 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c070300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0098d0 tx=0x7f3d4c02eed0 comp rx=0 tx=0).stop 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 1 -- 10.20.193.6:0/1446526682 shutdown_connections 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 1 --2- 10.20.193.6:0/1446526682 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c070300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 1 -- 10.20.193.6:0/1446526682 >> 10.20.193.6:0/1446526682 conn(0x7f3d5c0cd780 msgr2=0x7f3d5c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 1 -- 10.20.193.6:0/1446526682 shutdown_connections 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 1 -- 10.20.193.6:0/1446526682 wait complete. 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 10 monclient: build_initial_monmap 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.994+0000 7f3d63a16640 10 monclient: monmap: 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 librados: starting msgr at 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 librados: starting objecter 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 Processor -- start 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 -- start start 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 librados: setting wanted keys 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 librados: calling monclient init 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient: init 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c0bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): picked mon.noname-a con 0x7f3d5c06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c0c99a0 0x7f3d5c0bed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): picked mon.noname-c con 0x7f3d5c0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3d5c15a5e0 0x7f3d5c0bf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): picked mon.noname-b con 0x7f3d5c15a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c150520 con 0x7f3d5c06ff20 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c0c8a10 con 0x7f3d5c15a5e0 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1506a0 con 0x7f3d5c0c99a0 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d63a16640 10 monclient(hunting): authenticate will time out at 1295.723623s 2026-02-22T23:27:00.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d6178b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c0bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.995+0000 7f3d60f8a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c0c99a0 0x7f3d5c0bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c0bdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37402/0 (socket says 10.20.193.6:37402) 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 1 -- 10.20.193.6:0/3324938680 learned_addr learned my addr 10.20.193.6:0/3324938680 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 10 monclient(hunting): get_auth_request con 0x7f3d5c06ff20 auth_method 0 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d60f8a640 10 monclient(hunting): get_auth_request con 0x7f3d5c0c99a0 auth_method 0 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d60f8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d60f8a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d60f8a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3d5c15a5e0 0x7f3d5c0bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d60f8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d60f8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d60f8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d6178b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 10 monclient(hunting): get_auth_request con 0x7f3d5c15a5e0 auth_method 0 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:00.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.996+0000 7f3d61f8c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 10 monclient(hunting): handle_auth_done global_id 5045 payload 824 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 1 -- 10.20.193.6:0/3324938680 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3d5c15a5e0 msgr2=0x7f3d5c0bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3d5c15a5e0 0x7f3d5c0bf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 1 -- 10.20.193.6:0/3324938680 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3d5c06ff20 msgr2=0x7f3d5c0bdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c0bdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 10 monclient: _finish_hunting 0 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 1 monclient: found mon.noname-c 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.997+0000 7f3d60f8a640 1 -- 10.20.193.6:0/3324938680 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c0c1450 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.998+0000 7f3d60f8a640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c0c99a0 0x7f3d5c0bed30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000c480 tx=0x7f3d5000c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.998+0000 7f3d49ffb640 10 client.?.objecter ms_handle_connect 0x7f3d5c0c99a0 2026-02-22T23:27:01.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.998+0000 7f3d49ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d49ffb640 1 -- 10.20.193.6:0/3324938680 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50017070 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d49ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d49ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d49ffb640 10 monclient: dump: 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:01.001 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d49ffb640 10 monclient: _finish_auth 0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d49ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d49ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 5 monclient: authenticate success, global_id 5045 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 10 monclient: _renew_subs 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5c0c1730 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 10 client.5045.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 10 monclient: _renew_subs 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5c0c1cf0 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:00.999+0000 7f3d63a16640 1 librados: init done 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d63a16640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d63a16640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d63a16640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5c070300 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d49ffb640 1 -- 10.20.193.6:0/3324938680 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d500052d0 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d49ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d49ffb640 1 -- 10.20.193.6:0/3324938680 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50021060 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d49ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d49ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d4b7fe640 4 set_mon_vals no callback set 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.000+0000 7f3d49ffb640 10 monclient: dump: 2026-02-22T23:27:01.002 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:01.003 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:01.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.103+0000 7f3d49ffb640 1 -- 10.20.193.6:0/3324938680 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3d50024080 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.103+0000 7f3d49ffb640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3d2c078e30 0x7f3d2c07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:01.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d6178b640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3d2c078e30 0x7f3d2c07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:01.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d6178b640 10 monclient: get_auth_request con 0x7f3d2c078e30 auth_method 0 2026-02-22T23:27:01.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d6178b640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3d2c078e30 0x7f3d2c07b2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c02f400 tx=0x7f3d4c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:01.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d49ffb640 10 client.5045.objecter ms_handle_connect 0x7f3d2c078e30 2026-02-22T23:27:01.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d49ffb640 1 -- 10.20.193.6:0/3324938680 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f3d5005d420 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d49ffb640 10 client.5045.objecter ms_dispatch2 0x7f3d5c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:01.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d49ffb640 3 client.5045.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:01.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d49ffb640 3 client.5045.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:01.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.104+0000 7f3d49ffb640 20 client.5045.objecter dump_active .. 0 homeless 2026-02-22T23:27:01.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.111+0000 7f3d49ffb640 1 -- 10.20.193.6:0/3324938680 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3d500613e0 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.111+0000 7f3d49ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:01.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.111+0000 7f3d49ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:01.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.211+0000 7f3d63a16640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:01.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.211+0000 7f3d63a16640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:01.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.211+0000 7f3d63a16640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:01.214 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.211+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3d5c157ea0 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.214+0000 7f3d49ffb640 1 -- 10.20.193.6:0/3324938680 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f3d500656e0 con 0x7f3d5c0c99a0 2026-02-22T23:27:01.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.214+0000 7f3d49ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:01.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.214+0000 7f3d49ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:01.216 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:01.217 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:01.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.214+0000 7f3d63a16640 10 librados: watch_flush enter 2026-02-22T23:27:01.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.214+0000 7f3d63a16640 10 librados: watch_flush exit 2026-02-22T23:27:01.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.214+0000 7f3d63a16640 20 client.5045.objecter shutdown clearing up homeless session... 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.214+0000 7f3d63a16640 10 client.5045.objecter successfully canceled tick 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3d2c078e30 msgr2=0x7f3d2c07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3d2c078e30 0x7f3d2c07b2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c02f400 tx=0x7f3d4c033000 comp rx=0 tx=0).stop 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 10 monclient: shutdown 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c0c99a0 msgr2=0x7f3d5c0bed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c0c99a0 0x7f3d5c0bed30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000c480 tx=0x7f3d5000c950 comp rx=0 tx=0).stop 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 shutdown_connections 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3d2c078e30 0x7f3d2c07b2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3d5c15a5e0 0x7f3d5c0bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3d5c0c99a0 0x7f3d5c0bed30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:01.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 --2- 10.20.193.6:0/3324938680 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3d5c06ff20 0x7f3d5c0bdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:01.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 >> 10.20.193.6:0/3324938680 conn(0x7f3d5c0cd780 msgr2=0x7f3d5c14e500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:01.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 shutdown_connections 2026-02-22T23:27:01.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 -- 10.20.193.6:0/3324938680 wait complete. 2026-02-22T23:27:01.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:01.215+0000 7f3d63a16640 1 librados: shutdown 2026-02-22T23:27:04.226 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:04.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.293+0000 7f5eb0bf7640 10 monclient: get_monmap_and_config 2026-02-22T23:27:04.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.293+0000 7f5eb0bf7640 10 monclient: build_initial_monmap 2026-02-22T23:27:04.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.293+0000 7f5eb0bf7640 10 monclient: monmap: 2026-02-22T23:27:04.296 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:04.296 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:04.296 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:04.294962+0000 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:04.294962+0000 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.293+0000 7f5eb0bf7640 1 Processor -- start 2026-02-22T23:27:04.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.293+0000 7f5eb0bf7640 1 -- start start 2026-02-22T23:27:04.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.293+0000 7f5eb0bf7640 10 monclient: init 2026-02-22T23:27:04.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:04.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient: _add_conns ranks=[1,4,3,2,5,0] 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5eac06ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f5eac0c99a0 0x7f5eac070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient(hunting): picked mon.noname-b con 0x7f5eac0c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f5eac15a5e0 0x7f5eac14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient(hunting): picked mon.noname-a con 0x7f5eac15a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eac0c7e90 con 0x7f5eac06ff20 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac159a40 con 0x7f5eac15a5e0 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:04.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac0c8170 con 0x7f5eac0c99a0 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eb0bf7640 10 monclient(hunting): authenticate will time out at 1299.022307s 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eaad76640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f5eac15a5e0 0x7f5eac14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eaad76640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f5eac15a5e0 0x7f5eac14fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37412/0 (socket says 10.20.193.6:37412) 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eaad76640 1 -- 10.20.193.6:0/3024730905 learned_addr learned my addr 10.20.193.6:0/3024730905 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eaad76640 10 monclient(hunting): get_auth_request con 0x7f5eac15a5e0 auth_method 0 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eaad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eaad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5eaad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:04.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.294+0000 7f5ea9d74640 1 --2- 10.20.193.6:0/3024730905 >> v2:10.20.193.40:3300/0 conn(0x7f5eac0c99a0 0x7f5eac070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5eaad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5eaad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5eaad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5ea9d74640 10 monclient(hunting): get_auth_request con 0x7f5eac0c99a0 auth_method 0 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5ea9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5ea9d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5ea9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:04.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5ea9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5ea9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.295+0000 7f5ea9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 10 monclient(hunting): handle_auth_done global_id 4965 payload 274 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 1 -- 10.20.193.6:0/3024730905 >> v2:10.20.193.40:3300/0 conn(0x7f5eac0c99a0 msgr2=0x7f5eac070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 1 --2- 10.20.193.6:0/3024730905 >> v2:10.20.193.40:3300/0 conn(0x7f5eac0c99a0 0x7f5eac070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 1 -- 10.20.193.6:0/3024730905 >> v1:10.20.193.40:6789/0 conn(0x7f5eac06ff20 legacy=0x7f5eac159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 10 monclient: _finish_hunting 0 2026-02-22T23:27:04.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 1 monclient: found mon.noname-a 2026-02-22T23:27:04.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:04.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:04.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.296+0000 7f5eaad76640 1 -- 10.20.193.6:0/3024730905 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac0c8450 con 0x7f5eac15a5e0 2026-02-22T23:27:04.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.297+0000 7f5eaad76640 1 --2- 10.20.193.6:0/3024730905 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac15a5e0 0x7f5eac14fe10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009e50 tx=0x7f5e9c02f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7040d593a3c3e7 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:04.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 1 -- 10.20.193.6:0/3024730905 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c02fcc0 con 0x7f5eac15a5e0 2026-02-22T23:27:04.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: dump: 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:04.314 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: _finish_auth 0 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: _check_auth_tickets 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 1 -- 10.20.193.6:0/3024730905 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0373f0 con 0x7f5eac15a5e0 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5eb0bf7640 5 monclient: authenticate success, global_id 4965 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5eb0bf7640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 1 -- 10.20.193.6:0/3024730905 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c02fe60 con 0x7f5eac15a5e0 2026-02-22T23:27:04.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5ea9573640 10 monclient: dump: 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:04.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5eb0bf7640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.298+0000 7f5eb0bf7640 4 set_mon_vals no callback set 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 10 monclient: shutdown 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/3024730905 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac15a5e0 msgr2=0x7f5eac14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/3024730905 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac15a5e0 0x7f5eac14fe10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009e50 tx=0x7f5e9c02f2a0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/3024730905 shutdown_connections 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/3024730905 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac15a5e0 0x7f5eac14fe10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/3024730905 >> v2:10.20.193.40:3300/0 conn(0x7f5eac0c99a0 0x7f5eac070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/3024730905 >> 10.20.193.6:0/3024730905 conn(0x7f5eac0cd780 msgr2=0x7f5eac0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/3024730905 shutdown_connections 2026-02-22T23:27:04.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/3024730905 wait complete. 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 10 monclient: build_initial_monmap 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.299+0000 7f5eb0bf7640 10 monclient: monmap: 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:04.318 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 librados: starting msgr at 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 librados: starting objecter 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 Processor -- start 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 -- start start 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 librados: setting wanted keys 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 librados: calling monclient init 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient: init 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac06ff20 0x7f5eac156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): picked mon.noname-a con 0x7f5eac06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5eac0c99a0 0x7f5eac155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:04.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): picked mon.noname-b con 0x7f5eac0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5eac15a5e0 0x7f5eac0bcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): picked mon.noname-c con 0x7f5eac15a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac150350 con 0x7f5eac06ff20 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0c8a10 con 0x7f5eac0c99a0 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac1504d0 con 0x7f5eac15a5e0 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eb0bf7640 10 monclient(hunting): authenticate will time out at 1299.028531s 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eaa575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac06ff20 0x7f5eac156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:04.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eaa575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac06ff20 0x7f5eac156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37416/0 (socket says 10.20.193.6:37416) 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eaa575640 1 -- 10.20.193.6:0/4157773780 learned_addr learned my addr 10.20.193.6:0/4157773780 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eaa575640 10 monclient(hunting): get_auth_request con 0x7f5eac06ff20 auth_method 0 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eaa575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eaa575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5eaa575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.300+0000 7f5ea9d74640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5eac0c99a0 0x7f5eac155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5eac15a5e0 0x7f5eac0bcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5ea9d74640 10 monclient(hunting): get_auth_request con 0x7f5eac0c99a0 auth_method 0 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5ea9d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5ea9d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5ea9d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:04.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaa575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaa575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaa575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 10 monclient(hunting): get_auth_request con 0x7f5eac15a5e0 auth_method 0 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5ea9d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5ea9d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5ea9d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.301+0000 7f5eaad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:04.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 10 monclient(hunting): handle_auth_done global_id 4968 payload 824 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 1 -- 10.20.193.6:0/4157773780 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5eac15a5e0 msgr2=0x7f5eac0bcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5eac15a5e0 0x7f5eac0bcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 1 -- 10.20.193.6:0/4157773780 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5eac0c99a0 msgr2=0x7f5eac155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5eac0c99a0 0x7f5eac155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 10 monclient: _finish_hunting 0 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 1 monclient: found mon.noname-a 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.302+0000 7f5eaa575640 1 -- 10.20.193.6:0/4157773780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac157080 con 0x7f5eac06ff20 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.303+0000 7f5eaa575640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac06ff20 0x7f5eac156b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000c600 tx=0x7f5ea000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.303+0000 7f5e92ffd640 10 client.?.objecter ms_handle_connect 0x7f5eac06ff20 2026-02-22T23:27:04.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.303+0000 7f5e92ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 1 -- 10.20.193.6:0/4157773780 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0012830 con 0x7f5eac06ff20 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: dump: 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:04.324 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: _finish_auth 0 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 1 -- 10.20.193.6:0/4157773780 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea0017070 con 0x7f5eac06ff20 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 1 -- 10.20.193.6:0/4157773780 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0012c10 con 0x7f5eac06ff20 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5e92ffd640 10 monclient: dump: 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:04.325 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 5 monclient: authenticate success, global_id 4968 2026-02-22T23:27:04.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5ea8d72640 4 set_mon_vals no callback set 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 10 monclient: _renew_subs 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac0ba680 con 0x7f5eac06ff20 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 10 client.4968.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 10 monclient: _renew_subs 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac0bac40 con 0x7f5eac06ff20 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.305+0000 7f5eb0bf7640 1 librados: init done 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.306+0000 7f5eb0bf7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:04.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.306+0000 7f5eb0bf7640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:04.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.306+0000 7f5eb0bf7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:04.328 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.306+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eac150860 con 0x7f5eac06ff20 2026-02-22T23:27:04.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.409+0000 7f5e92ffd640 1 -- 10.20.193.6:0/4157773780 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5ea001c460 con 0x7f5eac06ff20 2026-02-22T23:27:04.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.409+0000 7f5e92ffd640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5e78078e00 0x7f5e7807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:04.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.409+0000 7f5ea9d74640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5e78078e00 0x7f5e7807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:04.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.409+0000 7f5ea9d74640 10 monclient: get_auth_request con 0x7f5e78078e00 auth_method 0 2026-02-22T23:27:04.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.410+0000 7f5e92ffd640 10 client.4968.objecter ms_handle_connect 0x7f5e78078e00 2026-02-22T23:27:04.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.410+0000 7f5ea9d74640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5e78078e00 0x7f5e7807b2c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0704a0 tx=0x7f5e94009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:04.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 1 -- 10.20.193.6:0/4157773780 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f5ea005e220 con 0x7f5eac06ff20 2026-02-22T23:27:04.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 10 client.4968.objecter ms_dispatch2 0x7f5eac000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:04.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 3 client.4968.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:04.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 3 client.4968.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:04.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 20 client.4968.objecter dump_active .. 0 homeless 2026-02-22T23:27:04.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 1 -- 10.20.193.6:0/4157773780 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5ea00621e0 con 0x7f5eac06ff20 2026-02-22T23:27:04.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:04.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.417+0000 7f5e92ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:04.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.518+0000 7f5eb0bf7640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:04.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.518+0000 7f5eb0bf7640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:04.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.518+0000 7f5eb0bf7640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:04.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.518+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5eac0c48e0 con 0x7f5eac06ff20 2026-02-22T23:27:04.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.520+0000 7f5e92ffd640 1 -- 10.20.193.6:0/4157773780 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f5ea00664e0 con 0x7f5eac06ff20 2026-02-22T23:27:04.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.520+0000 7f5e92ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:04.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.520+0000 7f5e92ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:04.524 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:04.524 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:04.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 10 librados: watch_flush enter 2026-02-22T23:27:04.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 10 librados: watch_flush exit 2026-02-22T23:27:04.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 20 client.4968.objecter shutdown clearing up homeless session... 2026-02-22T23:27:04.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 10 client.4968.objecter successfully canceled tick 2026-02-22T23:27:04.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5e78078e00 msgr2=0x7f5e7807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:04.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5e78078e00 0x7f5e7807b2c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0704a0 tx=0x7f5e94009290 comp rx=0 tx=0).stop 2026-02-22T23:27:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 10 monclient: shutdown 2026-02-22T23:27:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac06ff20 msgr2=0x7f5eac156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac06ff20 0x7f5eac156b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000c600 tx=0x7f5ea000cad0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 shutdown_connections 2026-02-22T23:27:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5e78078e00 0x7f5e7807b2c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5eac15a5e0 0x7f5eac0bcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5eac0c99a0 0x7f5eac155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 --2- 10.20.193.6:0/4157773780 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5eac06ff20 0x7f5eac156b40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:04.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 >> 10.20.193.6:0/4157773780 conn(0x7f5eac0cd780 msgr2=0x7f5eac06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:04.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 shutdown_connections 2026-02-22T23:27:04.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 -- 10.20.193.6:0/4157773780 wait complete. 2026-02-22T23:27:04.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:04.521+0000 7f5eb0bf7640 1 librados: shutdown 2026-02-22T23:27:07.534 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:07.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.602+0000 7f8a7d772640 10 monclient: get_monmap_and_config 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.602+0000 7f8a7d772640 10 monclient: build_initial_monmap 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.602+0000 7f8a7d772640 10 monclient: monmap: 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:07.604599+0000 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:07.604599+0000 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:07.606 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 1 Processor -- start 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 1 -- start start 2026-02-22T23:27:07.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient: init 2026-02-22T23:27:07.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:07.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient: _add_conns ranks=[1,0,5,2,3,4] 2026-02-22T23:27:07.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8a7806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:07.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8a780c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:07.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8a7815a5e0 0x7f8a78070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:07.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient(hunting): picked mon.noname-c con 0x7f8a7815a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:07.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a780c7e90 con 0x7f8a780c99a0 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a780c8170 con 0x7f8a7806ff20 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8a78159a40 con 0x7f8a7815a5e0 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.603+0000 7f8a7d772640 10 monclient(hunting): authenticate will time out at 1302.331968s 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.604+0000 7f8a767fc640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f8a780c99a0 0x7f8a7814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:56494/0 (socket says 10.20.193.6:56494) 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.604+0000 7f8a767fc640 1 -- 10.20.193.6:0/246166172 learned_addr learned my addr 10.20.193.6:0/246166172 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.604+0000 7f8a777fe640 1 --2- 10.20.193.6:0/246166172 >> v2:10.20.193.80:3300/0 conn(0x7f8a7815a5e0 0x7f8a78070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.604+0000 7f8a777fe640 10 monclient(hunting): get_auth_request con 0x7f8a7815a5e0 auth_method 0 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.604+0000 7f8a777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.604+0000 7f8a777fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:07.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.604+0000 7f8a777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a75ffb640 1 -- 10.20.193.6:0/246166172 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 395323033 0 0) 0x7f8a780c7e90 con 0x7f8a780c99a0 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a75ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a75ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a75ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a75ffb640 10 monclient(hunting): my global_id is 4971 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.605+0000 7f8a75ffb640 1 -- 10.20.193.6:0/246166172 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a64003620 con 0x7f8a780c99a0 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a75ffb640 1 -- 10.20.193.6:0/246166172 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1050950918 0 0) 0x7f8a780c8170 con 0x7f8a7806ff20 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a75ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:27:07.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a75ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a75ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a75ffb640 10 monclient(hunting): my global_id is 4930 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a75ffb640 1 -- 10.20.193.6:0/246166172 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a780c7e90 con 0x7f8a7806ff20 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 10 monclient(hunting): handle_auth_done global_id 5051 payload 274 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 1 -- 10.20.193.6:0/246166172 >> v1:10.20.193.40:6789/0 conn(0x7f8a7806ff20 legacy=0x7f8a78159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 1 -- 10.20.193.6:0/246166172 >> v1:10.20.193.6:6789/0 conn(0x7f8a780c99a0 legacy=0x7f8a7814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 1 monclient: found mon.noname-c 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.606+0000 7f8a777fe640 1 -- 10.20.193.6:0/246166172 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a780c8450 con 0x7f8a7815a5e0 2026-02-22T23:27:07.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.607+0000 7f8a777fe640 1 --2- 10.20.193.6:0/246166172 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a7815a5e0 0x7f8a78070300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009dc0 tx=0x7f8a6802f280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8815997463c7504f server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:07.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 1 -- 10.20.193.6:0/246166172 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6803c070 con 0x7f8a7815a5e0 2026-02-22T23:27:07.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:07.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: dump: 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:07.621 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: _finish_auth 0 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 1 -- 10.20.193.6:0/246166172 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a68037440 con 0x7f8a7815a5e0 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 5 monclient: authenticate success, global_id 5051 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 1 -- 10.20.193.6:0/246166172 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68037810 con 0x7f8a7815a5e0 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 4 set_mon_vals no callback set 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:07.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a75ffb640 10 monclient: dump: 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 10 monclient: shutdown 2026-02-22T23:27:07.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 1 -- 10.20.193.6:0/246166172 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a7815a5e0 msgr2=0x7f8a78070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 1 --2- 10.20.193.6:0/246166172 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a7815a5e0 0x7f8a78070300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009dc0 tx=0x7f8a6802f280 comp rx=0 tx=0).stop 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 1 -- 10.20.193.6:0/246166172 shutdown_connections 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 1 --2- 10.20.193.6:0/246166172 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a7815a5e0 0x7f8a78070300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 1 -- 10.20.193.6:0/246166172 >> 10.20.193.6:0/246166172 conn(0x7f8a780cd780 msgr2=0x7f8a780cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 1 -- 10.20.193.6:0/246166172 shutdown_connections 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.609+0000 7f8a7d772640 1 -- 10.20.193.6:0/246166172 wait complete. 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 10 monclient: build_initial_monmap 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 10 monclient: monmap: 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:07.624 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 1 librados: starting msgr at 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 1 librados: starting objecter 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 1 Processor -- start 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 1 -- start start 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 1 librados: setting wanted keys 2026-02-22T23:27:07.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 1 librados: calling monclient init 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.610+0000 7f8a7d772640 10 monclient: init 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8a7806ff20 0x7f8a780cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): picked mon.noname-a con 0x7f8a7806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a780c99a0 0x7f8a781f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): picked mon.noname-c con 0x7f8a780c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8a7815a5e0 0x7f8a781f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): picked mon.noname-b con 0x7f8a7815a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8a78150520 con 0x7f8a7806ff20 2026-02-22T23:27:07.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8a780c8a10 con 0x7f8a7815a5e0 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8a781506a0 con 0x7f8a780c99a0 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a7d772640 10 monclient(hunting): authenticate will time out at 1302.339246s 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a76ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8a7806ff20 0x7f8a780cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a767fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a780c99a0 0x7f8a781f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a777fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8a7815a5e0 0x7f8a781f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a76ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8a7806ff20 0x7f8a780cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37432/0 (socket says 10.20.193.6:37432) 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a76ffd640 1 -- 10.20.193.6:0/2521029917 learned_addr learned my addr 10.20.193.6:0/2521029917 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a76ffd640 10 monclient(hunting): get_auth_request con 0x7f8a7806ff20 auth_method 0 2026-02-22T23:27:07.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a76ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a76ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a76ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a767fc640 10 monclient(hunting): get_auth_request con 0x7f8a780c99a0 auth_method 0 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a767fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a767fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a767fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a777fe640 10 monclient(hunting): get_auth_request con 0x7f8a7815a5e0 auth_method 0 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a777fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a777fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.611+0000 7f8a777fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a767fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:07.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a767fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a767fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a76ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a76ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a76ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a777fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a777fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.612+0000 7f8a777fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 10 monclient(hunting): handle_auth_done global_id 5054 payload 824 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 1 -- 10.20.193.6:0/2521029917 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8a7815a5e0 msgr2=0x7f8a781f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:07.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8a7815a5e0 0x7f8a781f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 1 -- 10.20.193.6:0/2521029917 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8a7806ff20 msgr2=0x7f8a780cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8a7806ff20 0x7f8a780cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 10 monclient: _finish_hunting 0 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 1 monclient: found mon.noname-c 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.613+0000 7f8a767fc640 1 -- 10.20.193.6:0/2521029917 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a780bd2b0 con 0x7f8a780c99a0 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.614+0000 7f8a767fc640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a780c99a0 0x7f8a781f1570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c500 tx=0x7f8a6000c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.614+0000 7f8a577fe640 10 client.?.objecter ms_handle_connect 0x7f8a780c99a0 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.614+0000 7f8a577fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a577fe640 1 -- 10.20.193.6:0/2521029917 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60017070 con 0x7f8a780c99a0 2026-02-22T23:27:07.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a577fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a577fe640 10 monclient: dump: 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:07.631 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a577fe640 10 monclient: _finish_auth 0 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a577fe640 10 monclient: _check_auth_tickets 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a577fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 5 monclient: authenticate success, global_id 5054 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 10 monclient: _renew_subs 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a781f4a70 con 0x7f8a780c99a0 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 10 client.5054.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 10 monclient: _renew_subs 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a781f5030 con 0x7f8a780c99a0 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.615+0000 7f8a7d772640 1 librados: init done 2026-02-22T23:27:07.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a7d772640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a7d772640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a7d772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a78070300 con 0x7f8a780c99a0 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a577fe640 1 -- 10.20.193.6:0/2521029917 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a60005de0 con 0x7f8a780c99a0 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a577fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a577fe640 1 -- 10.20.193.6:0/2521029917 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60021060 con 0x7f8a780c99a0 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a577fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a577fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a757fa640 4 set_mon_vals no callback set 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.616+0000 7f8a577fe640 10 monclient: dump: 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:07.633 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.634 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:07.634 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:07.634 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:07.634 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:07.634 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:07.634 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:07.634 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:07.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.721+0000 7f8a577fe640 1 -- 10.20.193.6:0/2521029917 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8a60007270 con 0x7f8a780c99a0 2026-02-22T23:27:07.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.722+0000 7f8a577fe640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8a44078e00 0x7f8a4407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:07.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.722+0000 7f8a76ffd640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8a44078e00 0x7f8a4407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:07.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.722+0000 7f8a76ffd640 10 monclient: get_auth_request con 0x7f8a44078e00 auth_method 0 2026-02-22T23:27:07.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.722+0000 7f8a76ffd640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8a44078e00 0x7f8a4407b2c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c002410 tx=0x7f8a6c002bc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:07.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.722+0000 7f8a577fe640 10 client.5054.objecter ms_handle_connect 0x7f8a44078e00 2026-02-22T23:27:07.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.724+0000 7f8a577fe640 1 -- 10.20.193.6:0/2521029917 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f8a6005dbf0 con 0x7f8a780c99a0 2026-02-22T23:27:07.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.724+0000 7f8a577fe640 10 client.5054.objecter ms_dispatch2 0x7f8a78000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:07.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.724+0000 7f8a577fe640 3 client.5054.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:07.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.724+0000 7f8a577fe640 3 client.5054.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:07.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.724+0000 7f8a577fe640 20 client.5054.objecter dump_active .. 0 homeless 2026-02-22T23:27:07.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.730+0000 7f8a577fe640 1 -- 10.20.193.6:0/2521029917 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8a60061bb0 con 0x7f8a780c99a0 2026-02-22T23:27:07.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.731+0000 7f8a577fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:07.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.731+0000 7f8a577fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:07.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.831+0000 7f8a7d772640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:07.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.831+0000 7f8a7d772640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:07.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.831+0000 7f8a7d772640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:07.834 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.831+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8a780c4a10 con 0x7f8a780c99a0 2026-02-22T23:27:07.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.834+0000 7f8a577fe640 1 -- 10.20.193.6:0/2521029917 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8a60065eb0 con 0x7f8a780c99a0 2026-02-22T23:27:07.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.834+0000 7f8a577fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:07.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.834+0000 7f8a577fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:07.836 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:07.837 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:07.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 10 librados: watch_flush enter 2026-02-22T23:27:07.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 10 librados: watch_flush exit 2026-02-22T23:27:07.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 20 client.5054.objecter shutdown clearing up homeless session... 2026-02-22T23:27:07.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 10 client.5054.objecter successfully canceled tick 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8a44078e00 msgr2=0x7f8a4407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8a44078e00 0x7f8a4407b2c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c002410 tx=0x7f8a6c002bc0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 10 monclient: shutdown 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a780c99a0 msgr2=0x7f8a781f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a780c99a0 0x7f8a781f1570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c500 tx=0x7f8a6000c9d0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 shutdown_connections 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8a44078e00 0x7f8a4407b2c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8a7815a5e0 0x7f8a781f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8a780c99a0 0x7f8a781f1570 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 --2- 10.20.193.6:0/2521029917 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8a7806ff20 0x7f8a780cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:07.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 >> 10.20.193.6:0/2521029917 conn(0x7f8a780cd780 msgr2=0x7f8a7814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:07.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 shutdown_connections 2026-02-22T23:27:07.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 -- 10.20.193.6:0/2521029917 wait complete. 2026-02-22T23:27:07.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:07.835+0000 7f8a7d772640 1 librados: shutdown 2026-02-22T23:27:10.847 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:10.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.914+0000 7f1524a85640 10 monclient: get_monmap_and_config 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.914+0000 7f1524a85640 10 monclient: build_initial_monmap 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.914+0000 7f1524a85640 10 monclient: monmap: 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:10.915850+0000 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:10.915850+0000 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:10.917 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:10.918 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:10.918 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:10.918 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:10.918 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:10.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.914+0000 7f1524a85640 1 Processor -- start 2026-02-22T23:27:10.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.914+0000 7f1524a85640 1 -- start start 2026-02-22T23:27:10.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.914+0000 7f1524a85640 10 monclient: init 2026-02-22T23:27:10.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:10.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient: _add_conns ranks=[4,1,2,0,3,5] 2026-02-22T23:27:10.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f152006ff20 0x7f1520070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient(hunting): picked mon.noname-b con 0x7f152006ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f15200c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f152015a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15200c7e90 con 0x7f15200c99a0 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15200c8170 con 0x7f152015a5e0 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f1520159a40 con 0x7f152006ff20 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f1524a85640 10 monclient(hunting): authenticate will time out at 1305.643223s 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f151e575640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f152006ff20 0x7f1520070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f151ed76640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f152015a5e0 0x7f152014fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:37606/0 (socket says 10.20.193.6:37606) 2026-02-22T23:27:10.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f151ed76640 1 -- 10.20.193.6:0/2323204870 learned_addr learned my addr 10.20.193.6:0/2323204870 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f151e575640 10 monclient(hunting): get_auth_request con 0x7f152006ff20 auth_method 0 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f151e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f151e575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.915+0000 7f151e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.916+0000 7f151e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.916+0000 7f151e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.916+0000 7f151e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 10 monclient(hunting): handle_auth_done global_id 4936 payload 274 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 1 -- 10.20.193.6:0/2323204870 >> v1:10.20.193.80:6789/0 conn(0x7f152015a5e0 legacy=0x7f152014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 1 -- 10.20.193.6:0/2323204870 >> v1:10.20.193.40:6789/0 conn(0x7f15200c99a0 legacy=0x7f1520159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 10 monclient: _finish_hunting 0 2026-02-22T23:27:10.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 1 monclient: found mon.noname-b 2026-02-22T23:27:10.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:10.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:27:10.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.917+0000 7f151e575640 1 -- 10.20.193.6:0/2323204870 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15200c8450 con 0x7f152006ff20 2026-02-22T23:27:10.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.918+0000 7f151e575640 1 --2- 10.20.193.6:0/2323204870 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f1520070300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15140098d0 tx=0x7f151402ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd8c9702fc2507b server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:10.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 1 -- 10.20.193.6:0/2323204870 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151403c070 con 0x7f152006ff20 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: dump: 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:10.931 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: _finish_auth 0 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: _check_auth_tickets 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 1 -- 10.20.193.6:0/2323204870 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151402fb00 con 0x7f152006ff20 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 5 monclient: authenticate success, global_id 4936 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 1 -- 10.20.193.6:0/2323204870 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514037440 con 0x7f152006ff20 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:10.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f151d573640 10 monclient: dump: 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:10.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 4 set_mon_vals no callback set 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 10 monclient: shutdown 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 1 -- 10.20.193.6:0/2323204870 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 msgr2=0x7f1520070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 1 --2- 10.20.193.6:0/2323204870 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f1520070300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15140098d0 tx=0x7f151402ee80 comp rx=0 tx=0).stop 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 1 -- 10.20.193.6:0/2323204870 shutdown_connections 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 1 --2- 10.20.193.6:0/2323204870 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f1520070300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 1 -- 10.20.193.6:0/2323204870 >> 10.20.193.6:0/2323204870 conn(0x7f15200cd780 msgr2=0x7f15200cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 1 -- 10.20.193.6:0/2323204870 shutdown_connections 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.920+0000 7f1524a85640 1 -- 10.20.193.6:0/2323204870 wait complete. 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient: build_initial_monmap 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient: monmap: 2026-02-22T23:27:10.934 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 librados: starting msgr at 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 librados: starting objecter 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 Processor -- start 2026-02-22T23:27:10.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 -- start start 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 librados: setting wanted keys 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 librados: calling monclient init 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient: init 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f152014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): picked mon.noname-b con 0x7f152006ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f15200c99a0 0x7f15201f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): picked mon.noname-a con 0x7f15200c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f152015a5e0 0x7f15200cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): picked mon.noname-c con 0x7f152015a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:10.936 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1520150520 con 0x7f15200c99a0 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f15200c8a10 con 0x7f152006ff20 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f15201506a0 con 0x7f152015a5e0 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.921+0000 7f1524a85640 10 monclient(hunting): authenticate will time out at 1305.649911s 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f152014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151ed76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f152015a5e0 0x7f15200cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f15200c99a0 0x7f15201f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f152014f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:60008/0 (socket says 10.20.193.6:60008) 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 1 -- 10.20.193.6:0/2120297482 learned_addr learned my addr 10.20.193.6:0/2120297482 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:10.937 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 10 monclient(hunting): get_auth_request con 0x7f152006ff20 auth_method 0 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151ed76640 10 monclient(hunting): get_auth_request con 0x7f152015a5e0 auth_method 0 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151ed76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 10 monclient(hunting): get_auth_request con 0x7f15200c99a0 auth_method 0 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:10.938 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.922+0000 7f151ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 10 monclient(hunting): handle_auth_done global_id 4939 payload 824 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 1 -- 10.20.193.6:0/2120297482 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f152015a5e0 msgr2=0x7f15200cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:10.939 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f152015a5e0 0x7f15200cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 1 -- 10.20.193.6:0/2120297482 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f15200c99a0 msgr2=0x7f15201f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f15200c99a0 0x7f15201f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 10 monclient: _finish_hunting 0 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 1 monclient: found mon.noname-b 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.923+0000 7f151e575640 1 -- 10.20.193.6:0/2120297482 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152014e7e0 con 0x7f152006ff20 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.924+0000 7f151e575640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f152014f560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1514009130 tx=0x7f15140047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.924+0000 7f1506ffd640 10 client.?.objecter ms_handle_connect 0x7f152006ff20 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.924+0000 7f1506ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 1 -- 10.20.193.6:0/2120297482 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151403c040 con 0x7f152006ff20 2026-02-22T23:27:10.940 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: dump: 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:10.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: _finish_auth 0 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 5 monclient: authenticate success, global_id 4939 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 monclient: _renew_subs 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15200c3220 con 0x7f152006ff20 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 client.4939.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 monclient: _renew_subs 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15200c37e0 con 0x7f152006ff20 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 1 librados: init done 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 1 -- 10.20.193.6:0/2120297482 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151402fa60 con 0x7f152006ff20 2026-02-22T23:27:10.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 1 -- 10.20.193.6:0/2120297482 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514004430 con 0x7f152006ff20 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1506ffd640 10 monclient: dump: 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:10.943 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:10.944 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:10.944 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:10.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f151cd72640 4 set_mon_vals no callback set 2026-02-22T23:27:10.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:10.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:10.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:10.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:10.926+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1520070300 con 0x7f152006ff20 2026-02-22T23:27:11.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.031+0000 7f1506ffd640 1 -- 10.20.193.6:0/2120297482 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f151403f030 con 0x7f152006ff20 2026-02-22T23:27:11.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.032+0000 7f1506ffd640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f14ec078e30 0x7f14ec07b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:11.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.032+0000 7f151dd74640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f14ec078e30 0x7f14ec07b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:11.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.032+0000 7f151dd74640 10 monclient: get_auth_request con 0x7f14ec078e30 auth_method 0 2026-02-22T23:27:11.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.033+0000 7f151dd74640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f14ec078e30 0x7f14ec07b2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15200c0eb0 tx=0x7f15080091c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:11.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.033+0000 7f1506ffd640 10 client.4939.objecter ms_handle_connect 0x7f14ec078e30 2026-02-22T23:27:11.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.033+0000 7f1506ffd640 1 -- 10.20.193.6:0/2120297482 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f1514083670 con 0x7f152006ff20 2026-02-22T23:27:11.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.033+0000 7f1506ffd640 10 client.4939.objecter ms_dispatch2 0x7f1520000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:11.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.033+0000 7f1506ffd640 3 client.4939.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:11.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.033+0000 7f1506ffd640 3 client.4939.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:11.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.033+0000 7f1506ffd640 20 client.4939.objecter dump_active .. 0 homeless 2026-02-22T23:27:11.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.040+0000 7f1506ffd640 1 -- 10.20.193.6:0/2120297482 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1514050070 con 0x7f152006ff20 2026-02-22T23:27:11.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.040+0000 7f1506ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:11.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.040+0000 7f1506ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:11.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.143+0000 7f1524a85640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:11.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.143+0000 7f1524a85640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:11.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.143+0000 7f1524a85640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:11.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.143+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f15201583d0 con 0x7f152006ff20 2026-02-22T23:27:11.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.145+0000 7f1506ffd640 1 -- 10.20.193.6:0/2120297482 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f1514087630 con 0x7f152006ff20 2026-02-22T23:27:11.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.145+0000 7f1506ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:11.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.145+0000 7f1506ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:11.149 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:11.149 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:11.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 10 librados: watch_flush enter 2026-02-22T23:27:11.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 10 librados: watch_flush exit 2026-02-22T23:27:11.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 20 client.4939.objecter shutdown clearing up homeless session... 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 10 client.4939.objecter successfully canceled tick 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f14ec078e30 msgr2=0x7f14ec07b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f14ec078e30 0x7f14ec07b2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15200c0eb0 tx=0x7f15080091c0 comp rx=0 tx=0).stop 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 10 monclient: shutdown 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 msgr2=0x7f152014f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.146+0000 7f1524a85640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f152014f560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1514009130 tx=0x7f15140047c0 comp rx=0 tx=0).stop 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 shutdown_connections 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f14ec078e30 0x7f14ec07b2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f152015a5e0 0x7f15200cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f15200c99a0 0x7f15201f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:11.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 --2- 10.20.193.6:0/2120297482 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f152006ff20 0x7f152014f560 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:11.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 >> 10.20.193.6:0/2120297482 conn(0x7f15200cd780 msgr2=0x7f1520072480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:11.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 shutdown_connections 2026-02-22T23:27:11.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 -- 10.20.193.6:0/2120297482 wait complete. 2026-02-22T23:27:11.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:11.147+0000 7f1524a85640 1 librados: shutdown 2026-02-22T23:27:13.301 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:57.550 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.302 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:57.552 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.310 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:05:57.560 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.327 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:57.538 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.328 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:57.539 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.329 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:57.575 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.329 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:57.576 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.336 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:05:57.548 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:13.338 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:05:57.584 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:14.158 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:14.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.225+0000 7f782ce0b640 10 monclient: get_monmap_and_config 2026-02-22T23:27:14.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.225+0000 7f782ce0b640 10 monclient: build_initial_monmap 2026-02-22T23:27:14.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.225+0000 7f782ce0b640 10 monclient: monmap: 2026-02-22T23:27:14.230 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:14.230 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:14.227367+0000 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:14.227367+0000 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:14.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 1 Processor -- start 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 1 -- start start 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient: init 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient: _add_conns ranks=[0,5,1,3,2,4] 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f782806ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f78280c99a0 0x7f7828070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient(hunting): picked mon.noname-c con 0x7f78280c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f782815a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78280c7e90 con 0x7f782806ff20 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78280c8170 con 0x7f782815a5e0 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7828159a40 con 0x7f78280c99a0 2026-02-22T23:27:14.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.226+0000 7f782ce0b640 10 monclient(hunting): authenticate will time out at 1308.954734s 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7826575640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f782806ff20 0x7f7828159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:56598/0 (socket says 10.20.193.6:56598) 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7826575640 1 -- 10.20.193.6:0/1589575890 learned_addr learned my addr 10.20.193.6:0/1589575890 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 1 --2- 10.20.193.6:0/1589575890 >> v2:10.20.193.80:3300/0 conn(0x7f78280c99a0 0x7f7828070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 10 monclient(hunting): get_auth_request con 0x7f78280c99a0 auth_method 0 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:14.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.227+0000 7f7825d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 1 -- 10.20.193.6:0/1589575890 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1174369402 0 0) 0x7f78280c7e90 con 0x7f782806ff20 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): _negotiate 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): my global_id is 4980 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 1 -- 10.20.193.6:0/1589575890 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7814003620 con 0x7f782806ff20 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 1 -- 10.20.193.6:0/1589575890 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141205890 0 0) 0x7f78280c8170 con 0x7f782815a5e0 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): _negotiate 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:14.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 10 monclient(hunting): my global_id is 4942 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.228+0000 7f7825573640 1 -- 10.20.193.6:0/1589575890 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78280c7e90 con 0x7f782815a5e0 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 10 monclient(hunting): handle_auth_done global_id 5063 payload 274 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 1 -- 10.20.193.6:0/1589575890 >> v1:10.20.193.40:6789/0 conn(0x7f782815a5e0 legacy=0x7f782814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 1 -- 10.20.193.6:0/1589575890 >> v1:10.20.193.6:6789/0 conn(0x7f782806ff20 legacy=0x7f7828159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 10 monclient: _finish_hunting 0 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 1 monclient: found mon.noname-c 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.229+0000 7f7825d74640 1 -- 10.20.193.6:0/1589575890 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78280c8450 con 0x7f78280c99a0 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.230+0000 7f7825d74640 1 --2- 10.20.193.6:0/1589575890 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f78280c99a0 0x7f7828070300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f78100098d0 tx=0x7f781002eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad261bb50d34601e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 1 -- 10.20.193.6:0/1589575890 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781003c070 con 0x7f78280c99a0 2026-02-22T23:27:14.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: dump: 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:14.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: _finish_auth 0 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: _check_auth_tickets 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 1 -- 10.20.193.6:0/1589575890 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f781002fba0 con 0x7f78280c99a0 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 1 -- 10.20.193.6:0/1589575890 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810037440 con 0x7f78280c99a0 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 5 monclient: authenticate success, global_id 5063 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 4 set_mon_vals no callback set 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f7825573640 10 monclient: dump: 2026-02-22T23:27:14.243 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 10 monclient: shutdown 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:14.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 1 -- 10.20.193.6:0/1589575890 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f78280c99a0 msgr2=0x7f7828070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 1 --2- 10.20.193.6:0/1589575890 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f78280c99a0 0x7f7828070300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f78100098d0 tx=0x7f781002eed0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 1 -- 10.20.193.6:0/1589575890 shutdown_connections 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 1 --2- 10.20.193.6:0/1589575890 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f78280c99a0 0x7f7828070300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 1 -- 10.20.193.6:0/1589575890 >> 10.20.193.6:0/1589575890 conn(0x7f78280cd780 msgr2=0x7f78280cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 1 -- 10.20.193.6:0/1589575890 shutdown_connections 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.232+0000 7f782ce0b640 1 -- 10.20.193.6:0/1589575890 wait complete. 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 10 monclient: build_initial_monmap 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 10 monclient: monmap: 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:14.245 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 1 librados: starting msgr at 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 1 librados: starting objecter 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 1 Processor -- start 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 1 -- start start 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 1 librados: setting wanted keys 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 1 librados: calling monclient init 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.233+0000 7f782ce0b640 10 monclient: init 2026-02-22T23:27:14.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f782806ff20 0x7f7828154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): picked mon.noname-a con 0x7f782806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f78280c99a0 0x7f782814e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): picked mon.noname-b con 0x7f78280c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f782815a5e0 0x7f78280cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): picked mon.noname-c con 0x7f782815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7828150520 con 0x7f782806ff20 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f78280c8a10 con 0x7f78280c99a0 2026-02-22T23:27:14.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f78281506a0 con 0x7f782815a5e0 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f782ce0b640 10 monclient(hunting): authenticate will time out at 1308.962071s 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f782806ff20 0x7f7828154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f782815a5e0 0x7f78280cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7825d74640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f78280c99a0 0x7f782814e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f782806ff20 0x7f7828154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:48814/0 (socket says 10.20.193.6:48814) 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 1 -- 10.20.193.6:0/2111566479 learned_addr learned my addr 10.20.193.6:0/2111566479 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 10 monclient(hunting): get_auth_request con 0x7f782806ff20 auth_method 0 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:14.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 10 monclient(hunting): get_auth_request con 0x7f782815a5e0 auth_method 0 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7825d74640 10 monclient(hunting): get_auth_request con 0x7f78280c99a0 auth_method 0 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7825d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7825d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7825d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:14.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.234+0000 7f7826d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.235+0000 7f7825d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.235+0000 7f7825d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.235+0000 7f7825d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 10 monclient(hunting): handle_auth_done global_id 5066 payload 824 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 1 -- 10.20.193.6:0/2111566479 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f78280c99a0 msgr2=0x7f782814e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f78280c99a0 0x7f782814e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 1 -- 10.20.193.6:0/2111566479 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f782806ff20 msgr2=0x7f7828154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f782806ff20 0x7f7828154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 10 monclient: _finish_hunting 0 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 1 monclient: found mon.noname-c 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:14.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.236+0000 7f7826d76640 1 -- 10.20.193.6:0/2111566479 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78280cb170 con 0x7f782815a5e0 2026-02-22T23:27:14.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.237+0000 7f7826d76640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f782815a5e0 0x7f78280cc7e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f781800cb40 tx=0x7f781800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:14.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.237+0000 7f780effd640 10 client.?.objecter ms_handle_connect 0x7f782815a5e0 2026-02-22T23:27:14.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.237+0000 7f780effd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:14.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 1 -- 10.20.193.6:0/2111566479 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781800ad50 con 0x7f782815a5e0 2026-02-22T23:27:14.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:14.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:14.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: dump: 2026-02-22T23:27:14.253 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:14.253 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: _finish_auth 0 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:14.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 1 -- 10.20.193.6:0/2111566479 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818003a30 con 0x7f782815a5e0 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 1 -- 10.20.193.6:0/2111566479 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818003e20 con 0x7f782815a5e0 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f780effd640 10 monclient: dump: 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:14.255 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 5 monclient: authenticate success, global_id 5066 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 monclient: _renew_subs 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f7824d72640 4 set_mon_vals no callback set 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78280cb450 con 0x7f782815a5e0 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 client.5066.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 monclient: _renew_subs 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78280cba10 con 0x7f782815a5e0 2026-02-22T23:27:14.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 1 librados: init done 2026-02-22T23:27:14.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:14.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:14.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:14.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.239+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7828070300 con 0x7f782815a5e0 2026-02-22T23:27:14.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.251+0000 7f780effd640 1 -- 10.20.193.6:0/2111566479 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f78180050d0 con 0x7f782815a5e0 2026-02-22T23:27:14.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.252+0000 7f780effd640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f77f4078e50 0x7f77f407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:14.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.252+0000 7f7826575640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f77f4078e50 0x7f77f407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.252+0000 7f7826575640 10 monclient: get_auth_request con 0x7f77f4078e50 auth_method 0 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.252+0000 7f7826575640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f77f4078e50 0x7f77f407b310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f781c002410 tx=0x7f781c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.252+0000 7f780effd640 10 client.5066.objecter ms_handle_connect 0x7f77f4078e50 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.253+0000 7f780effd640 1 -- 10.20.193.6:0/2111566479 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f781805e5c0 con 0x7f782815a5e0 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.253+0000 7f780effd640 10 client.5066.objecter ms_dispatch2 0x7f7828000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.253+0000 7f780effd640 3 client.5066.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.253+0000 7f780effd640 3 client.5066.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:14.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.253+0000 7f780effd640 20 client.5066.objecter dump_active .. 0 homeless 2026-02-22T23:27:14.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.358+0000 7f780effd640 1 -- 10.20.193.6:0/2111566479 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7818062580 con 0x7f782815a5e0 2026-02-22T23:27:14.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.358+0000 7f780effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:14.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.358+0000 7f780effd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:14.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.458+0000 7f782ce0b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:14.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.458+0000 7f782ce0b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:14.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.458+0000 7f782ce0b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:14.461 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.458+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f78280c6e90 con 0x7f782815a5e0 2026-02-22T23:27:14.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f780effd640 1 -- 10.20.193.6:0/2111566479 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7818066880 con 0x7f782815a5e0 2026-02-22T23:27:14.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f780effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:14.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f780effd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:14.463 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:14.463 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 10 librados: watch_flush enter 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 10 librados: watch_flush exit 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 20 client.5066.objecter shutdown clearing up homeless session... 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 10 client.5066.objecter successfully canceled tick 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f77f4078e50 msgr2=0x7f77f407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f77f4078e50 0x7f77f407b310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f781c002410 tx=0x7f781c009040 comp rx=0 tx=0).stop 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 10 monclient: shutdown 2026-02-22T23:27:14.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f782815a5e0 msgr2=0x7f78280cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f782815a5e0 0x7f78280cc7e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f781800cb40 tx=0x7f781800a430 comp rx=0 tx=0).stop 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 shutdown_connections 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f77f4078e50 0x7f77f407b310 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f782815a5e0 0x7f78280cc7e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f78280c99a0 0x7f782814e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 --2- 10.20.193.6:0/2111566479 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f782806ff20 0x7f7828154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 >> 10.20.193.6:0/2111566479 conn(0x7f78280cd780 msgr2=0x7f78280cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 shutdown_connections 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 -- 10.20.193.6:0/2111566479 wait complete. 2026-02-22T23:27:14.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:14.461+0000 7f782ce0b640 1 librados: shutdown 2026-02-22T23:27:17.474 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:17.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.541+0000 7f8fffe3c640 10 monclient: get_monmap_and_config 2026-02-22T23:27:17.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.541+0000 7f8fffe3c640 10 monclient: build_initial_monmap 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.541+0000 7f8fffe3c640 10 monclient: monmap: 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:17.543463+0000 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:17.543463+0000 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:17.545 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:17.546 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:17.546 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:17.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 1 Processor -- start 2026-02-22T23:27:17.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 1 -- start start 2026-02-22T23:27:17.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient: init 2026-02-22T23:27:17.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:17.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient: _add_conns ranks=[3,2,0,5,1,4] 2026-02-22T23:27:17.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8ff806ff20 0x7f8ff8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient(hunting): picked mon.noname-a con 0x7f8ff806ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8ff80c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f8ff815a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff80c7e90 con 0x7f8ff815a5e0 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff80c8170 con 0x7f8ff80c99a0 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff8159a40 con 0x7f8ff806ff20 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.542+0000 7f8fffe3c640 10 monclient(hunting): authenticate will time out at 1312.270848s 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.543+0000 7f8ffdbb1640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8ff806ff20 0x7f8ff8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:17.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.543+0000 7f8ffd3b0640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f8ff80c99a0 0x7f8ff8159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:37616/0 (socket says 10.20.193.6:37616) 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.543+0000 7f8ffd3b0640 1 -- 10.20.193.6:0/1603543082 learned_addr learned my addr 10.20.193.6:0/1603543082 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.543+0000 7f8ffdbb1640 10 monclient(hunting): get_auth_request con 0x7f8ff806ff20 auth_method 0 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.543+0000 7f8ffdbb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.543+0000 7f8ffdbb1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.543+0000 7f8ffdbb1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffdbb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffdbb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffdbb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffcbaf640 1 -- 10.20.193.6:0/1603543082 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3842858674 0 0) 0x7f8ff80c8170 con 0x7f8ff80c99a0 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffcbaf640 10 monclient(hunting): _negotiate 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffcbaf640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffcbaf640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:17.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffcbaf640 10 monclient(hunting): my global_id is 5069 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.544+0000 7f8ffcbaf640 1 -- 10.20.193.6:0/1603543082 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fe0003620 con 0x7f8ff80c99a0 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 10 monclient(hunting): handle_auth_done global_id 4986 payload 274 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 1 -- 10.20.193.6:0/1603543082 >> v1:10.20.193.80:6789/0 conn(0x7f8ff80c99a0 legacy=0x7f8ff8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 1 -- 10.20.193.6:0/1603543082 >> v1:10.20.193.6:6789/0 conn(0x7f8ff815a5e0 legacy=0x7f8ff814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 10 monclient: _finish_hunting 0 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 1 monclient: found mon.noname-a 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:17.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.545+0000 7f8ffdbb1640 1 -- 10.20.193.6:0/1603543082 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff80c8450 con 0x7f8ff806ff20 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.547+0000 7f8ffdbb1640 1 --2- 10.20.193.6:0/1603543082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff806ff20 0x7f8ff8070300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098d0 tx=0x7f8fe802eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab915538f6da78a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 1 -- 10.20.193.6:0/1603543082 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe803c070 con 0x7f8ff806ff20 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: dump: 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:17.557 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: _finish_auth 0 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: _check_auth_tickets 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 1 -- 10.20.193.6:0/1603543082 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe802fba0 con 0x7f8ff806ff20 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 5 monclient: authenticate success, global_id 4986 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:17.558 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 1 -- 10.20.193.6:0/1603543082 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8037440 con 0x7f8ff806ff20 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8ffcbaf640 10 monclient: dump: 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:17.559 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 4 set_mon_vals no callback set 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 10 monclient: shutdown 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1603543082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff806ff20 msgr2=0x7f8ff8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1603543082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff806ff20 0x7f8ff8070300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098d0 tx=0x7f8fe802eed0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1603543082 shutdown_connections 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1603543082 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff806ff20 0x7f8ff8070300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1603543082 >> 10.20.193.6:0/1603543082 conn(0x7f8ff80cd780 msgr2=0x7f8ff80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1603543082 shutdown_connections 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.549+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1603543082 wait complete. 2026-02-22T23:27:17.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 10 monclient: build_initial_monmap 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 10 monclient: monmap: 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 1 librados: starting msgr at 2026-02-22T23:27:17.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 1 librados: starting objecter 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 1 Processor -- start 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 1 -- start start 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 1 librados: setting wanted keys 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 1 librados: calling monclient init 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.550+0000 7f8fffe3c640 10 monclient: init 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff806ff20 0x7f8ff814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): picked mon.noname-c con 0x7f8ff806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff80c99a0 0x7f8ff8154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): picked mon.noname-a con 0x7f8ff80c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff815a5e0 0x7f8ff8155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:17.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): picked mon.noname-b con 0x7f8ff815a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8150520 con 0x7f8ff80c99a0 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80c8a10 con 0x7f8ff815a5e0 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff81506a0 con 0x7f8ff806ff20 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8fffe3c640 10 monclient(hunting): authenticate will time out at 1312.279418s 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffd3b0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff80c99a0 0x7f8ff8154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffdbb1640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff806ff20 0x7f8ff814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffd3b0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff80c99a0 0x7f8ff8154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:48826/0 (socket says 10.20.193.6:48826) 2026-02-22T23:27:17.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffd3b0640 1 -- 10.20.193.6:0/1578293986 learned_addr learned my addr 10.20.193.6:0/1578293986 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffd3b0640 10 monclient(hunting): get_auth_request con 0x7f8ff80c99a0 auth_method 0 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffd3b0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffd3b0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffd3b0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffdbb1640 10 monclient(hunting): get_auth_request con 0x7f8ff806ff20 auth_method 0 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffdbb1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffdbb1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffdbb1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.551+0000 7f8ffe3b2640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff815a5e0 0x7f8ff8155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffd3b0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffd3b0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffd3b0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:17.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffdbb1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffdbb1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffdbb1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffe3b2640 10 monclient(hunting): get_auth_request con 0x7f8ff815a5e0 auth_method 0 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffe3b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffe3b2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffe3b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffe3b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffe3b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.552+0000 7f8ffe3b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 10 monclient(hunting): handle_auth_done global_id 4992 payload 824 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 1 -- 10.20.193.6:0/1578293986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff806ff20 msgr2=0x7f8ff814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff806ff20 0x7f8ff814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.565 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 1 -- 10.20.193.6:0/1578293986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff815a5e0 msgr2=0x7f8ff8155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff815a5e0 0x7f8ff8155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 10 monclient: _finish_hunting 0 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 1 monclient: found mon.noname-a 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.553+0000 7f8ffd3b0640 1 -- 10.20.193.6:0/1578293986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff8156d60 con 0x7f8ff80c99a0 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.554+0000 7f8ffd3b0640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff80c99a0 0x7f8ff8154f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c700 tx=0x7f8fec00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.554+0000 7f8fe67fc640 10 client.?.objecter ms_handle_connect 0x7f8ff80c99a0 2026-02-22T23:27:17.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.554+0000 7f8fe67fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:17.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 1 -- 10.20.193.6:0/1578293986 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec017070 con 0x7f8ff80c99a0 2026-02-22T23:27:17.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:17.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:17.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: dump: 2026-02-22T23:27:17.568 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:17.568 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:17.568 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: _finish_auth 0 2026-02-22T23:27:17.569 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 1 -- 10.20.193.6:0/1578293986 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec005bf0 con 0x7f8ff80c99a0 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 1 -- 10.20.193.6:0/1578293986 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec007070 con 0x7f8ff80c99a0 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fffe3c640 5 monclient: authenticate success, global_id 4992 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe7fff640 4 set_mon_vals no callback set 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fffe3c640 10 monclient: _renew_subs 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fffe3c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff8157040 con 0x7f8ff80c99a0 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.555+0000 7f8fe67fc640 10 monclient: dump: 2026-02-22T23:27:17.570 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 10 client.4992.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 10 monclient: _renew_subs 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:17.571 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff8154720 con 0x7f8ff80c99a0 2026-02-22T23:27:17.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 1 librados: init done 2026-02-22T23:27:17.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:17.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:17.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:17.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.556+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff8070300 con 0x7f8ff80c99a0 2026-02-22T23:27:17.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.567+0000 7f8fe67fc640 1 -- 10.20.193.6:0/1578293986 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8fec007210 con 0x7f8ff80c99a0 2026-02-22T23:27:17.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.567+0000 7f8fe67fc640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc8078e50 0x7f8fc807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:17.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.568+0000 7f8ffdbb1640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc8078e50 0x7f8fc807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:17.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.568+0000 7f8ffdbb1640 10 monclient: get_auth_request con 0x7f8fc8078e50 auth_method 0 2026-02-22T23:27:17.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.568+0000 7f8fe67fc640 10 client.4992.objecter ms_handle_connect 0x7f8fc8078e50 2026-02-22T23:27:17.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.568+0000 7f8ffdbb1640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc8078e50 0x7f8fc807b310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8039000 tx=0x7f8fe8033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:17.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.569+0000 7f8fe67fc640 1 -- 10.20.193.6:0/1578293986 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f8fec05d450 con 0x7f8ff80c99a0 2026-02-22T23:27:17.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.569+0000 7f8fe67fc640 10 client.4992.objecter ms_dispatch2 0x7f8ff8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:17.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.569+0000 7f8fe67fc640 3 client.4992.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:17.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.569+0000 7f8fe67fc640 3 client.4992.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:17.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.569+0000 7f8fe67fc640 20 client.4992.objecter dump_active .. 0 homeless 2026-02-22T23:27:17.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.664+0000 7f8fe67fc640 1 -- 10.20.193.6:0/1578293986 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8fec021020 con 0x7f8ff80c99a0 2026-02-22T23:27:17.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.664+0000 7f8fe67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:17.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.664+0000 7f8fe67fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:17.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.763+0000 7f8fffe3c640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:17.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.763+0000 7f8fffe3c640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:17.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.763+0000 7f8fffe3c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:17.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.763+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8ff81583f0 con 0x7f8ff80c99a0 2026-02-22T23:27:17.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.767+0000 7f8fe67fc640 1 -- 10.20.193.6:0/1578293986 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f8fec061410 con 0x7f8ff80c99a0 2026-02-22T23:27:17.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.767+0000 7f8fe67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:17.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.767+0000 7f8fe67fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:17.770 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:17.770 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 10 librados: watch_flush enter 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 10 librados: watch_flush exit 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 20 client.4992.objecter shutdown clearing up homeless session... 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 10 client.4992.objecter successfully canceled tick 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc8078e50 msgr2=0x7f8fc807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc8078e50 0x7f8fc807b310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8039000 tx=0x7f8fe8033000 comp rx=0 tx=0).stop 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 10 monclient: shutdown 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff80c99a0 msgr2=0x7f8ff8154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:17.772 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff80c99a0 0x7f8ff8154f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c700 tx=0x7f8fec00cbd0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 shutdown_connections 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc8078e50 0x7f8fc807b310 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff815a5e0 0x7f8ff8155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff80c99a0 0x7f8ff8154f20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 --2- 10.20.193.6:0/1578293986 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff806ff20 0x7f8ff814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 >> 10.20.193.6:0/1578293986 conn(0x7f8ff80cd780 msgr2=0x7f8ff814ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 shutdown_connections 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 -- 10.20.193.6:0/1578293986 wait complete. 2026-02-22T23:27:17.773 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:17.768+0000 7f8fffe3c640 1 librados: shutdown 2026-02-22T23:27:20.780 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:20.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 10 monclient: get_monmap_and_config 2026-02-22T23:27:20.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 10 monclient: build_initial_monmap 2026-02-22T23:27:20.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 10 monclient: monmap: 2026-02-22T23:27:20.853 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:20.853 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:20.853 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:20.851666+0000 2026-02-22T23:27:20.853 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:20.851666+0000 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 1 Processor -- start 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 1 -- start start 2026-02-22T23:27:20.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 10 monclient: init 2026-02-22T23:27:20.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:20.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 10 monclient: _add_conns ranks=[1,5,0,3,2,4] 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.851+0000 7f7fa2a96640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f7f9c06ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f7f9c0c99a0 0x7f7f9c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 10 monclient(hunting): picked mon.noname-c con 0x7f7f9c0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f7f9c15a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f9c0c7e90 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f9c0c8170 con 0x7f7f9c06ff20 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7f9c159a40 con 0x7f7f9c0c99a0 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:20.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa2a96640 10 monclient(hunting): authenticate will time out at 1315.579038s 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa100c640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f7f9c15a5e0 0x7f7f9c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:44288/0 (socket says 10.20.193.6:44288) 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7fa100c640 1 -- 10.20.193.6:0/2218957234 learned_addr learned my addr 10.20.193.6:0/2218957234 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7f93fff640 1 --2- 10.20.193.6:0/2218957234 >> v2:10.20.193.80:3300/0 conn(0x7f7f9c0c99a0 0x7f7f9c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7f93fff640 10 monclient(hunting): get_auth_request con 0x7f7f9c0c99a0 auth_method 0 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7f93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7f93fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.852+0000 7f7f93fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 1 -- 10.20.193.6:0/2218957234 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818645602 0 0) 0x7f7f9c0c7e90 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): _negotiate 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): my global_id is 4995 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 1 -- 10.20.193.6:0/2218957234 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f88003620 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 1 -- 10.20.193.6:0/2218957234 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231956647 0 0) 0x7f7f9c0c8170 con 0x7f7f9c06ff20 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): _negotiate 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 10 monclient(hunting): my global_id is 4951 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.853+0000 7f7f937fe640 1 -- 10.20.193.6:0/2218957234 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f9c0c7e90 con 0x7f7f9c06ff20 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 10 monclient(hunting): handle_auth_done global_id 5075 payload 274 2026-02-22T23:27:20.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 1 -- 10.20.193.6:0/2218957234 >> v1:10.20.193.40:6789/0 conn(0x7f7f9c06ff20 legacy=0x7f7f9c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:20.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 1 -- 10.20.193.6:0/2218957234 >> v1:10.20.193.6:6789/0 conn(0x7f7f9c15a5e0 legacy=0x7f7f9c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:20.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 10 monclient: _finish_hunting 0 2026-02-22T23:27:20.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 1 monclient: found mon.noname-c 2026-02-22T23:27:20.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:20.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:20.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.854+0000 7f7f93fff640 1 -- 10.20.193.6:0/2218957234 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9c0c8450 con 0x7f7f9c0c99a0 2026-02-22T23:27:20.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.855+0000 7f7f93fff640 1 --2- 10.20.193.6:0/2218957234 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 0x7f7f9c070300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f840098d0 tx=0x7f7f8402eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44b39703c23306e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 1 -- 10.20.193.6:0/2218957234 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8403c070 con 0x7f7f9c0c99a0 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: dump: 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.868 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: _finish_auth 0 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: _check_auth_tickets 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 1 -- 10.20.193.6:0/2218957234 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f8402fba0 con 0x7f7f9c0c99a0 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:20.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 1 -- 10.20.193.6:0/2218957234 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84037440 con 0x7f7f9c0c99a0 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 5 monclient: authenticate success, global_id 5075 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7f937fe640 10 monclient: dump: 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:20.870 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 4 set_mon_vals no callback set 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 10 monclient: shutdown 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2218957234 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 msgr2=0x7f7f9c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2218957234 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 0x7f7f9c070300 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7f840098d0 tx=0x7f7f8402eed0 comp rx=0 tx=0).stop 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2218957234 shutdown_connections 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2218957234 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 0x7f7f9c070300 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2218957234 >> 10.20.193.6:0/2218957234 conn(0x7f7f9c0cd780 msgr2=0x7f7f9c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2218957234 shutdown_connections 2026-02-22T23:27:20.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2218957234 wait complete. 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.857+0000 7f7fa2a96640 10 monclient: build_initial_monmap 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient: monmap: 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:20.872 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 librados: starting msgr at 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 librados: starting objecter 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 Processor -- start 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 -- start start 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 librados: setting wanted keys 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 librados: calling monclient init 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient: init 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7f9c06ff20 0x7f7f9c0c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): picked mon.noname-b con 0x7f7f9c06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 0x7f7f9c0c1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:20.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): picked mon.noname-c con 0x7f7f9c0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7f9c15a5e0 0x7f7f9c0bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): picked mon.noname-a con 0x7f7f9c15a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c150520 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c0c8a10 con 0x7f7f9c06ff20 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9c1506a0 con 0x7f7f9c0c99a0 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.858+0000 7f7fa2a96640 10 monclient(hunting): authenticate will time out at 1315.585757s 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7f9c15a5e0 0x7f7f9c0bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7f9c06ff20 0x7f7f9c0c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:20.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7f9c15a5e0 0x7f7f9c0bb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:53116/0 (socket says 10.20.193.6:53116) 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 1 -- 10.20.193.6:0/2624996180 learned_addr learned my addr 10.20.193.6:0/2624996180 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 10 monclient(hunting): get_auth_request con 0x7f7f9c15a5e0 auth_method 0 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 0x7f7f9c0c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 10 monclient(hunting): get_auth_request con 0x7f7f9c0c99a0 auth_method 0 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 10 monclient(hunting): get_auth_request con 0x7f7f9c06ff20 auth_method 0 2026-02-22T23:27:20.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa100c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7f93fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.859+0000 7f7fa080b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:20.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 10 monclient(hunting): handle_auth_done global_id 4998 payload 824 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 1 -- 10.20.193.6:0/2624996180 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 msgr2=0x7f7f9c0c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 0x7f7f9c0c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 1 -- 10.20.193.6:0/2624996180 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7f9c06ff20 msgr2=0x7f7f9c0c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7f9c06ff20 0x7f7f9c0c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 10 monclient: _finish_hunting 0 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 1 monclient: found mon.noname-a 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa100c640 1 -- 10.20.193.6:0/2624996180 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9c0c0a30 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.861+0000 7f7fa080b640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7f9c06ff20 0x7f7f9c0c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.862+0000 7f7fa100c640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7f9c15a5e0 0x7f7f9c0bb7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800caa0 tx=0x7f7f9800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:20.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.862+0000 7f7f917fa640 10 client.?.objecter ms_handle_connect 0x7f7f9c15a5e0 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.862+0000 7f7f917fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 1 -- 10.20.193.6:0/2624996180 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800ac40 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: dump: 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:20.878 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: _finish_auth 0 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: _check_auth_tickets 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 5 monclient: authenticate success, global_id 4998 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 10 monclient: _renew_subs 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 1 -- 10.20.193.6:0/2624996180 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9800ade0 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9c0be550 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:20.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 1 -- 10.20.193.6:0/2624996180 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98012620 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f917fa640 10 monclient: dump: 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:20.880 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7f92ffd640 4 set_mon_vals no callback set 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 10 client.4998.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 10 monclient: _renew_subs 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f9c0beb10 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.863+0000 7f7fa2a96640 1 librados: init done 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.864+0000 7f7fa2a96640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.864+0000 7f7fa2a96640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.864+0000 7f7fa2a96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:20.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.864+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f9c070300 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.875+0000 7f7f917fa640 1 -- 10.20.193.6:0/2624996180 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f7f980127c0 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.875+0000 7f7f917fa640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7f6c078e80 0x7f7f6c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.876+0000 7f7fa080b640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7f6c078e80 0x7f7f6c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.876+0000 7f7fa080b640 10 monclient: get_auth_request con 0x7f7f6c078e80 auth_method 0 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.876+0000 7f7fa080b640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7f6c078e80 0x7f7f6c07b340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c002410 tx=0x7f7f8c00e600 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.876+0000 7f7f917fa640 10 client.4998.objecter ms_handle_connect 0x7f7f6c078e80 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.877+0000 7f7f917fa640 1 -- 10.20.193.6:0/2624996180 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f7f9805e810 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.877+0000 7f7f917fa640 10 client.4998.objecter ms_dispatch2 0x7f7f9c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:20.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.877+0000 7f7f917fa640 3 client.4998.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:20.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.877+0000 7f7f917fa640 3 client.4998.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:20.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.877+0000 7f7f917fa640 20 client.4998.objecter dump_active .. 0 homeless 2026-02-22T23:27:20.976 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.975+0000 7f7f917fa640 1 -- 10.20.193.6:0/2624996180 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f7f980627d0 con 0x7f7f9c15a5e0 2026-02-22T23:27:20.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.975+0000 7f7f917fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:20.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:20.975+0000 7f7f917fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:21.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.077+0000 7f7fa2a96640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:21.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.077+0000 7f7fa2a96640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:21.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.077+0000 7f7fa2a96640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:21.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.077+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7f9c14f380 con 0x7f7f9c15a5e0 2026-02-22T23:27:21.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.080+0000 7f7f917fa640 1 -- 10.20.193.6:0/2624996180 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7f98066ad0 con 0x7f7f9c15a5e0 2026-02-22T23:27:21.081 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.080+0000 7f7f917fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:21.082 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.080+0000 7f7f917fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:21.082 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:21.082 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:21.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 10 librados: watch_flush enter 2026-02-22T23:27:21.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 10 librados: watch_flush exit 2026-02-22T23:27:21.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 20 client.4998.objecter shutdown clearing up homeless session... 2026-02-22T23:27:21.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 10 client.4998.objecter successfully canceled tick 2026-02-22T23:27:21.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7f6c078e80 msgr2=0x7f7f6c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:21.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7f6c078e80 0x7f7f6c07b340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c002410 tx=0x7f7f8c00e600 comp rx=0 tx=0).stop 2026-02-22T23:27:21.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 10 monclient: shutdown 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7f9c15a5e0 msgr2=0x7f7f9c0bb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7f9c15a5e0 0x7f7f9c0bb7e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800caa0 tx=0x7f7f9800cf70 comp rx=0 tx=0).stop 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 shutdown_connections 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f7f6c078e80 0x7f7f6c07b340 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f7f9c15a5e0 0x7f7f9c0bb7e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f7f9c0c99a0 0x7f7f9c0c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 --2- 10.20.193.6:0/2624996180 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f7f9c06ff20 0x7f7f9c0c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 >> 10.20.193.6:0/2624996180 conn(0x7f7f9c0cd780 msgr2=0x7f7f9c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 shutdown_connections 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 -- 10.20.193.6:0/2624996180 wait complete. 2026-02-22T23:27:21.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:21.081+0000 7f7fa2a96640 1 librados: shutdown 2026-02-22T23:27:24.093 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:24.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.160+0000 7f0c719e4640 10 monclient: get_monmap_and_config 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.160+0000 7f0c719e4640 10 monclient: build_initial_monmap 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.160+0000 7f0c719e4640 10 monclient: monmap: 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:24.161002+0000 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:24.161002+0000 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:24.162 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:24.163 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:24.163 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:24.163 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:24.163 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:24.163 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:24.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.160+0000 7f0c719e4640 1 Processor -- start 2026-02-22T23:27:24.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.160+0000 7f0c719e4640 1 -- start start 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.160+0000 7f0c719e4640 10 monclient: init 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient: _add_conns ranks=[1,0,3,5,2,4] 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0c6c06ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0c6c0c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f0c6c15a5e0 0x7f0c6c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient(hunting): picked mon.noname-a con 0x7f0c6c15a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c6c0c7e90 con 0x7f0c6c0c99a0 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c6c0c8170 con 0x7f0c6c06ff20 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f0c6c159a40 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c719e4640 10 monclient(hunting): authenticate will time out at 1318.888361s 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c6affd640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f0c6c06ff20 0x7f0c6c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:55166/0 (socket says 10.20.193.6:55166) 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c6affd640 1 -- 10.20.193.6:0/1689997394 learned_addr learned my addr 10.20.193.6:0/1689997394 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.161+0000 7f0c6b7fe640 1 --2- 10.20.193.6:0/1689997394 >> v2:10.20.193.6:3300/0 conn(0x7f0c6c15a5e0 0x7f0c6c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.162+0000 7f0c6b7fe640 10 monclient(hunting): get_auth_request con 0x7f0c6c15a5e0 auth_method 0 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.162+0000 7f0c6b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.162+0000 7f0c6b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.162+0000 7f0c6b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.162+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.162+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.162+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:24.167 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c69ffb640 1 -- 10.20.193.6:0/1689997394 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768979551 0 0) 0x7f0c6c0c8170 con 0x7f0c6c06ff20 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c69ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c69ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c69ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c69ffb640 10 monclient(hunting): my global_id is 4957 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c69ffb640 1 -- 10.20.193.6:0/1689997394 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c58003620 con 0x7f0c6c06ff20 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_done global_id 5001 payload 274 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 1 -- 10.20.193.6:0/1689997394 >> v1:10.20.193.40:6789/0 conn(0x7f0c6c06ff20 legacy=0x7f0c6c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 1 -- 10.20.193.6:0/1689997394 >> v1:10.20.193.6:6789/0 conn(0x7f0c6c0c99a0 legacy=0x7f0c6c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.163+0000 7f0c6b7fe640 1 -- 10.20.193.6:0/1689997394 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c0c8450 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.168 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.164+0000 7f0c6b7fe640 1 --2- 10.20.193.6:0/1689997394 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c070300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009dc0 tx=0x7f0c5c02f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccef1ebb9a01e9c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:24.173 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 1 -- 10.20.193.6:0/1689997394 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c03c070 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: dump: 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:24.174 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: _finish_auth 0 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 1 -- 10.20.193.6:0/1689997394 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c5c037440 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c719e4640 5 monclient: authenticate success, global_id 5001 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c719e4640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 1 -- 10.20.193.6:0/1689997394 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c037810 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c719e4640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c719e4640 4 set_mon_vals no callback set 2026-02-22T23:27:24.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.166+0000 7f0c69ffb640 10 monclient: dump: 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:24.176 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 10 monclient: shutdown 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 -- 10.20.193.6:0/1689997394 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 msgr2=0x7f0c6c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 --2- 10.20.193.6:0/1689997394 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c070300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009dc0 tx=0x7f0c5c02f230 comp rx=0 tx=0).stop 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 -- 10.20.193.6:0/1689997394 shutdown_connections 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 --2- 10.20.193.6:0/1689997394 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c070300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 -- 10.20.193.6:0/1689997394 >> 10.20.193.6:0/1689997394 conn(0x7f0c6c0cd780 msgr2=0x7f0c6c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 -- 10.20.193.6:0/1689997394 shutdown_connections 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 -- 10.20.193.6:0/1689997394 wait complete. 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 10 monclient: build_initial_monmap 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 10 monclient: monmap: 2026-02-22T23:27:24.177 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 librados: starting msgr at 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 librados: starting objecter 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.167+0000 7f0c719e4640 1 Processor -- start 2026-02-22T23:27:24.178 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 -- start start 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 librados: setting wanted keys 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 librados: calling monclient init 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient: init 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c6c06ff20 0x7f0c6c1f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): picked mon.noname-c con 0x7f0c6c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c6c0c99a0 0x7f0c6c0bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): picked mon.noname-b con 0x7f0c6c0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): picked mon.noname-a con 0x7f0c6c15a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:24.179 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c150520 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c0c8a10 con 0x7f0c6c0c99a0 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1506a0 con 0x7f0c6c06ff20 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c719e4640 10 monclient(hunting): authenticate will time out at 1318.895408s 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c1f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:53128/0 (socket says 10.20.193.6:53128) 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6b7fe640 1 -- 10.20.193.6:0/3959179722 learned_addr learned my addr 10.20.193.6:0/3959179722 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6b7fe640 10 monclient(hunting): get_auth_request con 0x7f0c6c15a5e0 auth_method 0 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:24.180 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6affd640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c6c06ff20 0x7f0c6c1f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.168+0000 7f0c6a7fc640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c6c0c99a0 0x7f0c6c0bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6affd640 10 monclient(hunting): get_auth_request con 0x7f0c6c06ff20 auth_method 0 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6a7fc640 10 monclient(hunting): get_auth_request con 0x7f0c6c0c99a0 auth_method 0 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:24.181 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.169+0000 7f0c6affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 10 monclient(hunting): handle_auth_done global_id 5004 payload 824 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 1 -- 10.20.193.6:0/3959179722 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c6c06ff20 msgr2=0x7f0c6c1f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c6c06ff20 0x7f0c6c1f1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 1 -- 10.20.193.6:0/3959179722 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c6c0c99a0 msgr2=0x7f0c6c0bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c6c0c99a0 0x7f0c6c0bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:24.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 1 monclient: found mon.noname-a 2026-02-22T23:27:24.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:24.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:24.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.170+0000 7f0c6b7fe640 1 -- 10.20.193.6:0/3959179722 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c1f4f40 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.171+0000 7f0c6b7fe640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c1f4a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c03a040 tx=0x7f0c5c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:24.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.171+0000 7f0c43fff640 10 client.?.objecter ms_handle_connect 0x7f0c6c15a5e0 2026-02-22T23:27:24.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.171+0000 7f0c43fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 1 -- 10.20.193.6:0/3959179722 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c047020 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: dump: 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.185 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: _finish_auth 0 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: _check_auth_tickets 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 1 -- 10.20.193.6:0/3959179722 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c5c042670 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 5 monclient: authenticate success, global_id 5004 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 1 -- 10.20.193.6:0/3959179722 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c03c040 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 monclient: _renew_subs 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c697fa640 4 set_mon_vals no callback set 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c1f5210 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c43fff640 10 monclient: dump: 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:24.187 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 client.5004.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 monclient: _renew_subs 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c1f57d0 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 1 librados: init done 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:24.188 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.173+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c070300 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.188+0000 7f0c43fff640 1 -- 10.20.193.6:0/3959179722 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0c5c042a60 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.189+0000 7f0c43fff640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c3c078e50 0x7f0c3c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:24.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.189+0000 7f0c6affd640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c3c078e50 0x7f0c3c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:24.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.189+0000 7f0c6affd640 10 monclient: get_auth_request con 0x7f0c3c078e50 auth_method 0 2026-02-22T23:27:24.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.189+0000 7f0c6affd640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c3c078e50 0x7f0c3c07b310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0c60007b60 tx=0x7f0c60007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:24.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.189+0000 7f0c43fff640 10 client.5004.objecter ms_handle_connect 0x7f0c3c078e50 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 1 -- 10.20.193.6:0/3959179722 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f0c5c042d80 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 10 client.5004.objecter ms_dispatch2 0x7f0c6c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 3 client.5004.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 3 client.5004.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 20 client.5004.objecter dump_active .. 0 homeless 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 1 -- 10.20.193.6:0/3959179722 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0c5c051030 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:24.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.286+0000 7f0c43fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:24.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.388+0000 7f0c719e4640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:24.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.388+0000 7f0c719e4640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:24.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.388+0000 7f0c719e4640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:24.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.388+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0c6c1573f0 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.391+0000 7f0c43fff640 1 -- 10.20.193.6:0/3959179722 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f0c5c087e10 con 0x7f0c6c15a5e0 2026-02-22T23:27:24.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.391+0000 7f0c43fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:24.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.391+0000 7f0c43fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:24.393 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:24.393 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:24.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 10 librados: watch_flush enter 2026-02-22T23:27:24.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 10 librados: watch_flush exit 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 20 client.5004.objecter shutdown clearing up homeless session... 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 10 client.5004.objecter successfully canceled tick 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c3c078e50 msgr2=0x7f0c3c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c3c078e50 0x7f0c3c07b310 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0c60007b60 tx=0x7f0c60007ab0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 10 monclient: shutdown 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 msgr2=0x7f0c6c1f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c1f4a00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c03a040 tx=0x7f0c5c004b40 comp rx=0 tx=0).stop 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 shutdown_connections 2026-02-22T23:27:24.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0c3c078e50 0x7f0c3c07b310 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0c6c15a5e0 0x7f0c6c1f4a00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0c6c0c99a0 0x7f0c6c0bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 --2- 10.20.193.6:0/3959179722 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0c6c06ff20 0x7f0c6c1f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:24.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 >> 10.20.193.6:0/3959179722 conn(0x7f0c6c0cd780 msgr2=0x7f0c6c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:24.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 shutdown_connections 2026-02-22T23:27:24.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 -- 10.20.193.6:0/3959179722 wait complete. 2026-02-22T23:27:24.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:24.392+0000 7f0c719e4640 1 librados: shutdown 2026-02-22T23:27:27.403 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:27.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.472+0000 7f4cce7d6640 10 monclient: get_monmap_and_config 2026-02-22T23:27:27.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.472+0000 7f4cce7d6640 10 monclient: build_initial_monmap 2026-02-22T23:27:27.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.472+0000 7f4cce7d6640 10 monclient: monmap: 2026-02-22T23:27:27.474 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:27.473285+0000 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:27.473285+0000 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:27.475 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:27.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 1 Processor -- start 2026-02-22T23:27:27.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 1 -- start start 2026-02-22T23:27:27.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient: init 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient: _add_conns ranks=[2,0,1,4,3,5] 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f4cc806ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4cc80c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4cc815a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc80c7e90 con 0x7f4cc80c99a0 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc80c8170 con 0x7f4cc815a5e0 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cc80c8450 con 0x7f4cc806ff20 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:27.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.473+0000 7f4cce7d6640 10 monclient(hunting): authenticate will time out at 1322.200641s 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.474+0000 7f4cc77fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f4cc80c99a0 0x7f4cc814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:44298/0 (socket says 10.20.193.6:44298) 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.474+0000 7f4cc77fe640 1 -- 10.20.193.6:0/4085902422 learned_addr learned my addr 10.20.193.6:0/4085902422 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68168332 0 0) 0x7f4cc80c7e90 con 0x7f4cc80c99a0 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _negotiate 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): my global_id is 5007 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cac003620 con 0x7f4cc80c99a0 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2868221164 0 0) 0x7f4cc80c8170 con 0x7f4cc815a5e0 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _negotiate 2026-02-22T23:27:27.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): my global_id is 4963 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cc80c7e90 con 0x7f4cc815a5e0 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729327243 0 0) 0x7f4cc80c8450 con 0x7f4cc806ff20 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _negotiate 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 10 monclient(hunting): my global_id is 5084 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.475+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cc80c8170 con 0x7f4cc806ff20 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1237222141 0 0) 0x7f4cac003620 con 0x7f4cc80c99a0 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 >> v1:10.20.193.80:6789/0 conn(0x7f4cc806ff20 legacy=0x7f4cc8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 >> v1:10.20.193.40:6789/0 conn(0x7f4cc815a5e0 legacy=0x7f4cc80c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc80c8730 con 0x7f4cc80c99a0 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f4cb8002f00 con 0x7f4cc80c99a0 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: dump: 2026-02-22T23:27:27.484 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 >> v1:10.20.193.6:6789/0 conn(0x7f4cc80c99a0 legacy=0x7f4cc814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cac004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): picked mon.b con 0x7f4cac003fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cac004e40 0x7f4cac0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): picked mon.c con 0x7f4cac004e40 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cac009ab0 0x7f4cac00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): picked mon.a con 0x7f4cac009ab0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac00c570 con 0x7f4cac009ab0 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac00c6f0 con 0x7f4cac003fe0 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:27.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac00c8f0 con 0x7f4cac004e40 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc6ffd640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.477+0000 7f4cc7fff640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cac004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc7fff640 10 monclient(hunting): get_auth_request con 0x7f4cac003fe0 auth_method 0 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc7fff640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cac009ab0 0x7f4cac00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc7fff640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:27:27.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 10 monclient(hunting): get_auth_request con 0x7f4cac009ab0 auth_method 0 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cc77fe640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cac004e40 0x7f4cac0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.478+0000 7f4cccd4c640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc77fe640 10 monclient(hunting): get_auth_request con 0x7f4cac004e40 auth_method 0 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 10 monclient(hunting): handle_auth_done global_id 5007 payload 293 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 1 -- 10.20.193.6:0/4085902422 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cac004e40 msgr2=0x7f4cac0052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:27.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cac004e40 0x7f4cac0052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 1 -- 10.20.193.6:0/4085902422 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cac009ab0 msgr2=0x7f4cac00bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cac009ab0 0x7f4cac00bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 10 monclient: _finish_hunting 0 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 1 monclient: found mon.b 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cccd4c640 1 -- 10.20.193.6:0/4085902422 reap_dead start 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 1 -- 10.20.193.6:0/4085902422 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac003620 con 0x7f4cac003fe0 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 10 monclient: _finish_auth 0 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cc7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:27.489 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cce7d6640 5 monclient: authenticate success, global_id 5007 2026-02-22T23:27:27.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.479+0000 7f4cce7d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:27.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.480+0000 7f4cc7fff640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cac004900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40031a0 tx=0x7f4cb402ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9cd52dee6c570f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:27.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.481+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4002dc0 con 0x7f4cac003fe0 2026-02-22T23:27:27.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.481+0000 7f4cc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:27.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.481+0000 7f4cc6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:27:27.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.481+0000 7f4cc6ffd640 10 monclient: dump: 2026-02-22T23:27:27.496 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.481+0000 7f4cce7d6640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:27.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb40052d0 con 0x7f4cac003fe0 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cc6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cc6ffd640 1 -- 10.20.193.6:0/4085902422 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb402f720 con 0x7f4cac003fe0 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cc6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cc6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cc6ffd640 10 monclient: dump: 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:27.498 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 4 set_mon_vals no callback set 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 10 monclient: shutdown 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 1 -- 10.20.193.6:0/4085902422 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 msgr2=0x7f4cac004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cac004900 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40031a0 tx=0x7f4cb402ef00 comp rx=0 tx=0).stop 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 1 -- 10.20.193.6:0/4085902422 shutdown_connections 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/4085902422 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cac004900 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 1 -- 10.20.193.6:0/4085902422 >> 10.20.193.6:0/4085902422 conn(0x7f4cc80cd780 msgr2=0x7f4cc80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:27.499 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 1 -- 10.20.193.6:0/4085902422 shutdown_connections 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 1 -- 10.20.193.6:0/4085902422 wait complete. 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 10 monclient: build_initial_monmap 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.482+0000 7f4cce7d6640 10 monclient: monmap: 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:27.500 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 librados: starting msgr at 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 librados: starting objecter 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 Processor -- start 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 -- start start 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 librados: setting wanted keys 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 librados: calling monclient init 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient: init 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cc81f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): picked mon.noname-b con 0x7f4cac003fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cc81453d0 0x7f4cc80c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:27.501 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): picked mon.noname-c con 0x7f4cc81453d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cc8145850 0x7f4cc81f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): picked mon.noname-a con 0x7f4cc8145850 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8070300 con 0x7f4cc8145850 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8159a40 con 0x7f4cac003fe0 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8070480 con 0x7f4cc81453d0 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cce7d6640 10 monclient(hunting): authenticate will time out at 1322.210591s 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cccd4c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cc8145850 0x7f4cc81f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.483+0000 7f4cc7fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cc81f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:27.502 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cc8145850 0x7f4cc81f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:53134/0 (socket says 10.20.193.6:53134) 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 1 -- 10.20.193.6:0/1548589285 learned_addr learned my addr 10.20.193.6:0/1548589285 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 10 monclient(hunting): get_auth_request con 0x7f4cc8145850 auth_method 0 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cc81453d0 0x7f4cc80c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc7fff640 10 monclient(hunting): get_auth_request con 0x7f4cac003fe0 auth_method 0 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc7fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:27.503 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cccd4c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 10 monclient(hunting): get_auth_request con 0x7f4cc81453d0 auth_method 0 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc7fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc7fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc7fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.484+0000 7f4cc77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 10 monclient(hunting): handle_auth_done global_id 5010 payload 824 2026-02-22T23:27:27.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 1 -- 10.20.193.6:0/1548589285 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cc81453d0 msgr2=0x7f4cc80c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cc81453d0 0x7f4cc80c10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 1 -- 10.20.193.6:0/1548589285 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 msgr2=0x7f4cc81f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cc81f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 10 monclient: _finish_hunting 0 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 1 monclient: found mon.noname-a 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.485+0000 7f4cccd4c640 1 -- 10.20.193.6:0/1548589285 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc81f2610 con 0x7f4cc8145850 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.486+0000 7f4cccd4c640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cc8145850 0x7f4cc81f2b80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80c9110 tx=0x7f4cbc00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.486+0000 7f4cc4ff9640 10 client.?.objecter ms_handle_connect 0x7f4cc8145850 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.486+0000 7f4cc4ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:27.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 1 -- 10.20.193.6:0/1548589285 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc005920 con 0x7f4cc8145850 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: dump: 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:27.506 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: _finish_auth 0 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 1 -- 10.20.193.6:0/1548589285 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc00fe20 con 0x7f4cc8145850 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 1 -- 10.20.193.6:0/1548589285 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc004020 con 0x7f4cc8145850 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc4ff9640 10 monclient: dump: 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.507 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cc5ffb640 4 set_mon_vals no callback set 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 5 monclient: authenticate success, global_id 5010 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 monclient: _renew_subs 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc80c1c80 con 0x7f4cc8145850 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 client.5010.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 monclient: _renew_subs 2026-02-22T23:27:27.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:27.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc80c2240 con 0x7f4cc8145850 2026-02-22T23:27:27.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 1 librados: init done 2026-02-22T23:27:27.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:27.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:27.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:27.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.488+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc8159ed0 con 0x7f4cc8145850 2026-02-22T23:27:27.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.499+0000 7f4cc4ff9640 1 -- 10.20.193.6:0/1548589285 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4cbc0041c0 con 0x7f4cc8145850 2026-02-22T23:27:27.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.500+0000 7f4cc4ff9640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4c98079020 0x7f4c9807b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:27.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.500+0000 7f4cc7fff640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4c98079020 0x7f4c9807b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:27.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.500+0000 7f4cc7fff640 10 monclient: get_auth_request con 0x7f4c98079020 auth_method 0 2026-02-22T23:27:27.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.501+0000 7f4cc4ff9640 1 -- 10.20.193.6:0/1548589285 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f4cbc0044e0 con 0x7f4cc8145850 2026-02-22T23:27:27.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.501+0000 7f4cc4ff9640 10 client.5010.objecter ms_dispatch2 0x7f4cc8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:27.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.501+0000 7f4cc4ff9640 3 client.5010.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:27.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.501+0000 7f4cc4ff9640 3 client.5010.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:27.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.501+0000 7f4cc7fff640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4c98079020 0x7f4c9807b4e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4037000 tx=0x7f4cb4009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:27.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.501+0000 7f4cc4ff9640 20 client.5010.objecter dump_active .. 0 homeless 2026-02-22T23:27:27.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.501+0000 7f4cc4ff9640 10 client.5010.objecter ms_handle_connect 0x7f4c98079020 2026-02-22T23:27:27.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.597+0000 7f4cc4ff9640 1 -- 10.20.193.6:0/1548589285 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4cbc062310 con 0x7f4cc8145850 2026-02-22T23:27:27.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.597+0000 7f4cc4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:27.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.597+0000 7f4cc4ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.700+0000 7f4cce7d6640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:27.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.700+0000 7f4cce7d6640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.700+0000 7f4cce7d6640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:27.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.700+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4cc8158050 con 0x7f4cc8145850 2026-02-22T23:27:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.702+0000 7f4cc4ff9640 1 -- 10.20.193.6:0/1548589285 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4cbc066610 con 0x7f4cc8145850 2026-02-22T23:27:27.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.702+0000 7f4cc4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:27.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.702+0000 7f4cc4ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:27.704 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:27.704 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:27.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 10 librados: watch_flush enter 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 10 librados: watch_flush exit 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 20 client.5010.objecter shutdown clearing up homeless session... 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 10 client.5010.objecter successfully canceled tick 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4c98079020 msgr2=0x7f4c9807b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4c98079020 0x7f4c9807b4e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4037000 tx=0x7f4cb4009030 comp rx=0 tx=0).stop 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 10 monclient: shutdown 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cc8145850 msgr2=0x7f4cc81f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cc8145850 0x7f4cc81f2b80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80c9110 tx=0x7f4cbc00f5a0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 shutdown_connections 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4c98079020 0x7f4c9807b4e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4cc8145850 0x7f4cc81f2b80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4cc81453d0 0x7f4cc80c10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 --2- 10.20.193.6:0/1548589285 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4cac003fe0 0x7f4cc81f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 >> 10.20.193.6:0/1548589285 conn(0x7f4cc80cd780 msgr2=0x7f4cc80cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 shutdown_connections 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 -- 10.20.193.6:0/1548589285 wait complete. 2026-02-22T23:27:27.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:27.703+0000 7f4cce7d6640 1 librados: shutdown 2026-02-22T23:27:30.714 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.783+0000 7f4f89674640 10 monclient: get_monmap_and_config 2026-02-22T23:27:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.783+0000 7f4f89674640 10 monclient: build_initial_monmap 2026-02-22T23:27:30.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.783+0000 7f4f89674640 10 monclient: monmap: 2026-02-22T23:27:30.785 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:30.784366+0000 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:30.784366+0000 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:30.786 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 1 Processor -- start 2026-02-22T23:27:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 1 -- start start 2026-02-22T23:27:30.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient: init 2026-02-22T23:27:30.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient: _add_conns ranks=[0,5,1,4,2,3] 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f4f8406ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f4f840c99a0 0x7f4f84070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient(hunting): picked mon.noname-c con 0x7f4f840c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f4f8415a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f840c7e90 con 0x7f4f8406ff20 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f840c8170 con 0x7f4f8415a5e0 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4f84159a40 con 0x7f4f840c99a0 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.784+0000 7f4f89674640 10 monclient(hunting): authenticate will time out at 1325.511743s 2026-02-22T23:27:30.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f82ffd640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f4f8406ff20 0x7f4f84159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:60454/0 (socket says 10.20.193.6:60454) 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f82ffd640 1 -- 10.20.193.6:0/3995412006 learned_addr learned my addr 10.20.193.6:0/3995412006 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 1 --2- 10.20.193.6:0/3995412006 >> v2:10.20.193.80:3300/0 conn(0x7f4f840c99a0 0x7f4f84070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 10 monclient(hunting): get_auth_request con 0x7f4f840c99a0 auth_method 0 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.785+0000 7f4f827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 1 -- 10.20.193.6:0/3995412006 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 748213148 0 0) 0x7f4f840c7e90 con 0x7f4f8406ff20 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:27:30.791 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): my global_id is 5013 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 1 -- 10.20.193.6:0/3995412006 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f70003620 con 0x7f4f8406ff20 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 1 -- 10.20.193.6:0/3995412006 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 234007747 0 0) 0x7f4f840c8170 con 0x7f4f8415a5e0 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 10 monclient(hunting): my global_id is 4969 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f81ffb640 1 -- 10.20.193.6:0/3995412006 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f840c7e90 con 0x7f4f8415a5e0 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 10 monclient(hunting): handle_auth_done global_id 5090 payload 274 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 1 -- 10.20.193.6:0/3995412006 >> v1:10.20.193.40:6789/0 conn(0x7f4f8415a5e0 legacy=0x7f4f8414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 1 -- 10.20.193.6:0/3995412006 >> v1:10.20.193.6:6789/0 conn(0x7f4f8406ff20 legacy=0x7f4f84159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:30.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 10 monclient: _finish_hunting 0 2026-02-22T23:27:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 1 monclient: found mon.noname-c 2026-02-22T23:27:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.786+0000 7f4f827fc640 1 -- 10.20.193.6:0/3995412006 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f840c8450 con 0x7f4f840c99a0 2026-02-22T23:27:30.793 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.787+0000 7f4f827fc640 1 --2- 10.20.193.6:0/3995412006 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f840c99a0 0x7f4f84070300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0098d0 tx=0x7f4f6c02eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0a4e9d1537d018 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 1 -- 10.20.193.6:0/3995412006 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c03c070 con 0x7f4f840c99a0 2026-02-22T23:27:30.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: dump: 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:30.800 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: _finish_auth 0 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 1 -- 10.20.193.6:0/3995412006 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6c02fba0 con 0x7f4f840c99a0 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f89674640 5 monclient: authenticate success, global_id 5090 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 1 -- 10.20.193.6:0/3995412006 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c037440 con 0x7f4f840c99a0 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f89674640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:30.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f89674640 4 set_mon_vals no callback set 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f81ffb640 10 monclient: dump: 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:30.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f89674640 10 monclient: shutdown 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f89674640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f89674640 1 -- 10.20.193.6:0/3995412006 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f840c99a0 msgr2=0x7f4f84070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.789+0000 7f4f89674640 1 --2- 10.20.193.6:0/3995412006 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f840c99a0 0x7f4f84070300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0098d0 tx=0x7f4f6c02eed0 comp rx=0 tx=0).stop 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 -- 10.20.193.6:0/3995412006 shutdown_connections 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 --2- 10.20.193.6:0/3995412006 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f840c99a0 0x7f4f84070300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 -- 10.20.193.6:0/3995412006 >> 10.20.193.6:0/3995412006 conn(0x7f4f840cd780 msgr2=0x7f4f840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 -- 10.20.193.6:0/3995412006 shutdown_connections 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 -- 10.20.193.6:0/3995412006 wait complete. 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 10 monclient: build_initial_monmap 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 10 monclient: monmap: 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:30.803 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 librados: starting msgr at 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 librados: starting objecter 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 Processor -- start 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 -- start start 2026-02-22T23:27:30.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 librados: setting wanted keys 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 1 librados: calling monclient init 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.790+0000 7f4f89674640 10 monclient: init 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4f8406ff20 0x7f4f84154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): picked mon.noname-a con 0x7f4f8406ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4f840c99a0 0x7f4f8414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): picked mon.noname-b con 0x7f4f840c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f8415a5e0 0x7f4f840cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): picked mon.noname-c con 0x7f4f8415a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84150520 con 0x7f4f8406ff20 2026-02-22T23:27:30.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f4f840c8a10 con 0x7f4f840c99a0 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841506a0 con 0x7f4f8415a5e0 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f89674640 10 monclient(hunting): authenticate will time out at 1325.518235s 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4f8406ff20 0x7f4f84154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f827fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4f840c99a0 0x7f4f8414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4f8406ff20 0x7f4f84154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:52620/0 (socket says 10.20.193.6:52620) 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 1 -- 10.20.193.6:0/1635210449 learned_addr learned my addr 10.20.193.6:0/1635210449 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 10 monclient(hunting): get_auth_request con 0x7f4f8406ff20 auth_method 0 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:30.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f837fe640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f8415a5e0 0x7f4f840cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f837fe640 10 monclient(hunting): get_auth_request con 0x7f4f8415a5e0 auth_method 0 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f837fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f837fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f837fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f827fc640 10 monclient(hunting): get_auth_request con 0x7f4f840c99a0 auth_method 0 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f827fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f827fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f827fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:30.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.791+0000 7f4f82ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.792+0000 7f4f837fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.792+0000 7f4f837fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.792+0000 7f4f837fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.792+0000 7f4f827fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.792+0000 7f4f827fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.792+0000 7f4f827fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 10 monclient(hunting): handle_auth_done global_id 5016 payload 824 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 1 -- 10.20.193.6:0/1635210449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f8415a5e0 msgr2=0x7f4f840cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f8415a5e0 0x7f4f840cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 1 -- 10.20.193.6:0/1635210449 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4f840c99a0 msgr2=0x7f4f8414e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4f840c99a0 0x7f4f8414e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:27:30.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 1 monclient: found mon.noname-a 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.793+0000 7f4f82ffd640 1 -- 10.20.193.6:0/1635210449 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f840cb170 con 0x7f4f8406ff20 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.794+0000 7f4f82ffd640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4f8406ff20 0x7f4f84154550 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c760 tx=0x7f4f7800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.794+0000 7f4f637fe640 10 client.?.objecter ms_handle_connect 0x7f4f8406ff20 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.794+0000 7f4f637fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 1 -- 10.20.193.6:0/1635210449 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78017070 con 0x7f4f8406ff20 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: dump: 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:30.809 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: _finish_auth 0 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: _check_auth_tickets 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 1 -- 10.20.193.6:0/1635210449 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f78005bd0 con 0x7f4f8406ff20 2026-02-22T23:27:30.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 5 monclient: authenticate success, global_id 5016 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 10 monclient: _renew_subs 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f840cb450 con 0x7f4f8406ff20 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 1 -- 10.20.193.6:0/1635210449 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78007070 con 0x7f4f8406ff20 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f817fa640 4 set_mon_vals no callback set 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f637fe640 10 monclient: dump: 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.811 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 10 client.5016.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 10 monclient: _renew_subs 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f840cba10 con 0x7f4f8406ff20 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.795+0000 7f4f89674640 1 librados: init done 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.796+0000 7f4f89674640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:30.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.796+0000 7f4f89674640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:30.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.796+0000 7f4f89674640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:30.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.796+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f84070300 con 0x7f4f8406ff20 2026-02-22T23:27:30.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.807+0000 7f4f637fe640 1 -- 10.20.193.6:0/1635210449 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f4f78007210 con 0x7f4f8406ff20 2026-02-22T23:27:30.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.808+0000 7f4f637fe640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4f50078e50 0x7f4f5007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:30.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.808+0000 7f4f827fc640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4f50078e50 0x7f4f5007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:30.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.808+0000 7f4f827fc640 10 monclient: get_auth_request con 0x7f4f50078e50 auth_method 0 2026-02-22T23:27:30.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.809+0000 7f4f827fc640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4f50078e50 0x7f4f5007b310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c039000 tx=0x7f4f6c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:30.814 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.809+0000 7f4f637fe640 10 client.5016.objecter ms_handle_connect 0x7f4f50078e50 2026-02-22T23:27:30.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.809+0000 7f4f637fe640 1 -- 10.20.193.6:0/1635210449 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f4f7805d4d0 con 0x7f4f8406ff20 2026-02-22T23:27:30.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.809+0000 7f4f637fe640 10 client.5016.objecter ms_dispatch2 0x7f4f84000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:30.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.809+0000 7f4f637fe640 3 client.5016.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:30.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.809+0000 7f4f637fe640 3 client.5016.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:30.815 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.809+0000 7f4f637fe640 20 client.5016.objecter dump_active .. 0 homeless 2026-02-22T23:27:30.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.906+0000 7f4f637fe640 1 -- 10.20.193.6:0/1635210449 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f4f78021020 con 0x7f4f8406ff20 2026-02-22T23:27:30.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.906+0000 7f4f637fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:30.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:30.906+0000 7f4f637fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:31.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.008+0000 7f4f89674640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:31.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.008+0000 7f4f89674640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:31.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.008+0000 7f4f89674640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:31.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.008+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4f840c6e90 con 0x7f4f8406ff20 2026-02-22T23:27:31.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.011+0000 7f4f637fe640 1 -- 10.20.193.6:0/1635210449 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f4f78061490 con 0x7f4f8406ff20 2026-02-22T23:27:31.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.011+0000 7f4f637fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:31.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.011+0000 7f4f637fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:31.013 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:31.013 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:31.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 10 librados: watch_flush enter 2026-02-22T23:27:31.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 10 librados: watch_flush exit 2026-02-22T23:27:31.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 20 client.5016.objecter shutdown clearing up homeless session... 2026-02-22T23:27:31.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 10 client.5016.objecter successfully canceled tick 2026-02-22T23:27:31.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4f50078e50 msgr2=0x7f4f5007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:31.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4f50078e50 0x7f4f5007b310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c039000 tx=0x7f4f6c033000 comp rx=0 tx=0).stop 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 10 monclient: shutdown 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4f8406ff20 msgr2=0x7f4f84154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4f8406ff20 0x7f4f84154550 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c760 tx=0x7f4f7800cc30 comp rx=0 tx=0).stop 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 shutdown_connections 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f4f50078e50 0x7f4f5007b310 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f4f8415a5e0 0x7f4f840cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f4f840c99a0 0x7f4f8414e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 --2- 10.20.193.6:0/1635210449 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f4f8406ff20 0x7f4f84154550 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 >> 10.20.193.6:0/1635210449 conn(0x7f4f840cd780 msgr2=0x7f4f840cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 shutdown_connections 2026-02-22T23:27:31.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 -- 10.20.193.6:0/1635210449 wait complete. 2026-02-22T23:27:31.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:31.012+0000 7f4f89674640 1 librados: shutdown 2026-02-22T23:27:33.326 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:17.576 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.328 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:17.578 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.335 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:17.585 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.355 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:17.566 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.355 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:17.601 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.356 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:17.567 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.356 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:17.603 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.363 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:17.574 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:33.364 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:17.611 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:34.023 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:34.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.092+0000 7fd531308640 10 monclient: get_monmap_and_config 2026-02-22T23:27:34.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.092+0000 7fd531308640 10 monclient: build_initial_monmap 2026-02-22T23:27:34.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.092+0000 7fd531308640 10 monclient: monmap: 2026-02-22T23:27:34.094 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:34.094 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:34.094 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:34.093312+0000 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:34.093312+0000 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:34.095 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:34.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 1 Processor -- start 2026-02-22T23:27:34.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 1 -- start start 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient: init 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient: _add_conns ranks=[0,2,5,4,1,3] 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fd52c1527d0 addr v1:10.20.193.6:6789/0 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd52c152bb0 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd52c0c6190 0x7fd52c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient(hunting): picked mon.noname-c con 0x7fd52c0c6190 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd52c06e300 con 0x7fd52c1527d0 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd52c06e5e0 con 0x7fd52c152bb0 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:34.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd52c06f1f0 con 0x7fd52c0c6190 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.093+0000 7fd531308640 10 monclient(hunting): authenticate will time out at 1328.820796s 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.095+0000 7fd52b7fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd52c0c6190 0x7fd52c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.095+0000 7fd52affd640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fd52c1527d0 0x7fd52c06da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:60460/0 (socket says 10.20.193.6:60460) 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.095+0000 7fd52affd640 1 -- 10.20.193.6:0/229974530 learned_addr learned my addr 10.20.193.6:0/229974530 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.095+0000 7fd52b7fe640 10 monclient(hunting): get_auth_request con 0x7fd52c0c6190 auth_method 0 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.095+0000 7fd52b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.095+0000 7fd52b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.095+0000 7fd52b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:34.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:34.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd529ffb640 1 -- 10.20.193.6:0/229974530 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960491020 0 0) 0x7fd52c06e300 con 0x7fd52c1527d0 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd529ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd529ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd529ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd529ffb640 10 monclient(hunting): my global_id is 5019 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.096+0000 7fd529ffb640 1 -- 10.20.193.6:0/229974530 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd518003620 con 0x7fd52c1527d0 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_done global_id 5096 payload 274 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 1 -- 10.20.193.6:0/229974530 >> v1:10.20.193.80:6789/0 conn(0x7fd52c152bb0 legacy=0x7fd52c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 1 -- 10.20.193.6:0/229974530 >> v1:10.20.193.6:6789/0 conn(0x7fd52c1527d0 legacy=0x7fd52c06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 1 monclient: found mon.noname-c 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:34.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:34.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.097+0000 7fd52b7fe640 1 -- 10.20.193.6:0/229974530 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c06e8c0 con 0x7fd52c0c6190 2026-02-22T23:27:34.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.100+0000 7fd52b7fe640 1 --2- 10.20.193.6:0/229974530 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c0c6190 0x7fd52c0c8e60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd52c06f370 tx=0x7fd51c02f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e3bd09019060500 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:34.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 1 -- 10.20.193.6:0/229974530 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c03c070 con 0x7fd52c0c6190 2026-02-22T23:27:34.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:34.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: dump: 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:34.108 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: _finish_auth 0 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 1 -- 10.20.193.6:0/229974530 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c037440 con 0x7fd52c0c6190 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 5 monclient: authenticate success, global_id 5096 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 1 -- 10.20.193.6:0/229974530 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c037810 con 0x7fd52c0c6190 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:34.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd529ffb640 10 monclient: dump: 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:34.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 4 set_mon_vals no callback set 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 10 monclient: shutdown 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 1 -- 10.20.193.6:0/229974530 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c0c6190 msgr2=0x7fd52c0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 1 --2- 10.20.193.6:0/229974530 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c0c6190 0x7fd52c0c8e60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd52c06f370 tx=0x7fd51c02f230 comp rx=0 tx=0).stop 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 1 -- 10.20.193.6:0/229974530 shutdown_connections 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 1 --2- 10.20.193.6:0/229974530 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c0c6190 0x7fd52c0c8e60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 1 -- 10.20.193.6:0/229974530 >> 10.20.193.6:0/229974530 conn(0x7fd52c0cd780 msgr2=0x7fd52c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 1 -- 10.20.193.6:0/229974530 shutdown_connections 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.102+0000 7fd531308640 1 -- 10.20.193.6:0/229974530 wait complete. 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient: build_initial_monmap 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient: monmap: 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:34.111 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 librados: starting msgr at 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 librados: starting objecter 2026-02-22T23:27:34.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 Processor -- start 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 -- start start 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 librados: setting wanted keys 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 librados: calling monclient init 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient: init 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd52c0c6190 0x7fd52c0c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): picked mon.noname-b con 0x7fd52c0c6190 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd52c1527d0 0x7fd52c158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): picked mon.noname-a con 0x7fd52c1527d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c152bb0 0x7fd52c14ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:34.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): picked mon.noname-c con 0x7fd52c152bb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0c93a0 con 0x7fd52c1527d0 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c06ee80 con 0x7fd52c0c6190 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c0c9520 con 0x7fd52c152bb0 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.103+0000 7fd531308640 10 monclient(hunting): authenticate will time out at 1328.830979s 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd52c1527d0 0x7fd52c158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c152bb0 0x7fd52c14ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd52c1527d0 0x7fd52c158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:52624/0 (socket says 10.20.193.6:52624) 2026-02-22T23:27:34.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 1 -- 10.20.193.6:0/127291514 learned_addr learned my addr 10.20.193.6:0/127291514 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 10 monclient(hunting): get_auth_request con 0x7fd52c1527d0 auth_method 0 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52affd640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd52c0c6190 0x7fd52c0c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 10 monclient(hunting): get_auth_request con 0x7fd52c152bb0 auth_method 0 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52affd640 10 monclient(hunting): get_auth_request con 0x7fd52c0c6190 auth_method 0 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:34.115 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52a7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.104+0000 7fd52affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.105+0000 7fd52affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 10 monclient(hunting): handle_auth_done global_id 5102 payload 824 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 1 -- 10.20.193.6:0/127291514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd52c0c6190 msgr2=0x7fd52c0c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:34.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd52c0c6190 0x7fd52c0c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 1 -- 10.20.193.6:0/127291514 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd52c1527d0 msgr2=0x7fd52c158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:34.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd52c1527d0 0x7fd52c158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:34.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 1 monclient: found mon.noname-c 2026-02-22T23:27:34.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:34.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:34.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.106+0000 7fd52b7fe640 1 -- 10.20.193.6:0/127291514 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c14df40 con 0x7fd52c152bb0 2026-02-22T23:27:34.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.107+0000 7fd52b7fe640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c152bb0 0x7fd52c14ef90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0c5660 tx=0x7fd51c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:34.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.107+0000 7fd50b7fe640 10 client.?.objecter ms_handle_connect 0x7fd52c152bb0 2026-02-22T23:27:34.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.107+0000 7fd50b7fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:34.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 1 -- 10.20.193.6:0/127291514 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c03c050 con 0x7fd52c152bb0 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: dump: 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:34.120 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 1 -- 10.20.193.6:0/127291514 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c048070 con 0x7fd52c152bb0 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 5 monclient: authenticate success, global_id 5102 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 10 monclient: _renew_subs 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c14e220 con 0x7fd52c152bb0 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:34.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 1 -- 10.20.193.6:0/127291514 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c041a50 con 0x7fd52c152bb0 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd5297fa640 4 set_mon_vals no callback set 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd50b7fe640 10 monclient: dump: 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:34.122 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 10 client.5102.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 10 monclient: _renew_subs 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c14e7e0 con 0x7fd52c152bb0 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.108+0000 7fd531308640 1 librados: init done 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.109+0000 7fd531308640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:34.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.109+0000 7fd531308640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:34.124 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.109+0000 7fd531308640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:34.124 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.109+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c14ea20 con 0x7fd52c152bb0 2026-02-22T23:27:34.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.120+0000 7fd50b7fe640 1 -- 10.20.193.6:0/127291514 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd51c041bf0 con 0x7fd52c152bb0 2026-02-22T23:27:34.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.121+0000 7fd50b7fe640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd4f8078e50 0x7fd4f807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:34.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.121+0000 7fd52affd640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd4f8078e50 0x7fd4f807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:34.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.121+0000 7fd52affd640 10 monclient: get_auth_request con 0x7fd4f8078e50 auth_method 0 2026-02-22T23:27:34.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.122+0000 7fd50b7fe640 10 client.5102.objecter ms_handle_connect 0x7fd4f8078e50 2026-02-22T23:27:34.125 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.122+0000 7fd52affd640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd4f8078e50 0x7fd4f807b310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd520005980 tx=0x7fd520007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:34.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.122+0000 7fd50b7fe640 1 -- 10.20.193.6:0/127291514 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fd51c084350 con 0x7fd52c152bb0 2026-02-22T23:27:34.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.122+0000 7fd50b7fe640 10 client.5102.objecter ms_dispatch2 0x7fd52c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:34.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.122+0000 7fd50b7fe640 3 client.5102.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:34.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.122+0000 7fd50b7fe640 3 client.5102.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:34.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.123+0000 7fd50b7fe640 20 client.5102.objecter dump_active .. 0 homeless 2026-02-22T23:27:34.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.221+0000 7fd50b7fe640 1 -- 10.20.193.6:0/127291514 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fd52c14ea20 con 0x7fd52c152bb0 2026-02-22T23:27:34.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.221+0000 7fd50b7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:34.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.221+0000 7fd50b7fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:34.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.322+0000 7fd531308640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:34.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.322+0000 7fd531308640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:34.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.322+0000 7fd531308640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:34.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.322+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd52c0c9ed0 con 0x7fd52c152bb0 2026-02-22T23:27:34.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.343+0000 7fd50b7fe640 1 -- 10.20.193.6:0/127291514 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fd52c0c9ed0 con 0x7fd52c152bb0 2026-02-22T23:27:34.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.343+0000 7fd50b7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:34.344 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.343+0000 7fd50b7fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:34.345 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:34.345 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:34.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.344+0000 7fd531308640 10 librados: watch_flush enter 2026-02-22T23:27:34.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 10 librados: watch_flush exit 2026-02-22T23:27:34.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 20 client.5102.objecter shutdown clearing up homeless session... 2026-02-22T23:27:34.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 10 client.5102.objecter successfully canceled tick 2026-02-22T23:27:34.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd4f8078e50 msgr2=0x7fd4f807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:34.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd4f8078e50 0x7fd4f807b310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd520005980 tx=0x7fd520007dd0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 10 monclient: shutdown 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c152bb0 msgr2=0x7fd52c14ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c152bb0 0x7fd52c14ef90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd52c0c5660 tx=0x7fd51c004b40 comp rx=0 tx=0).stop 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 shutdown_connections 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd4f8078e50 0x7fd4f807b310 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd52c152bb0 0x7fd52c14ef90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd52c1527d0 0x7fd52c158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 --2- 10.20.193.6:0/127291514 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd52c0c6190 0x7fd52c0c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 >> 10.20.193.6:0/127291514 conn(0x7fd52c0cd780 msgr2=0x7fd52c1515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 shutdown_connections 2026-02-22T23:27:34.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 -- 10.20.193.6:0/127291514 wait complete. 2026-02-22T23:27:34.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:34.345+0000 7fd531308640 1 librados: shutdown 2026-02-22T23:27:37.356 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.422+0000 7fc782485640 10 monclient: get_monmap_and_config 2026-02-22T23:27:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.422+0000 7fc782485640 10 monclient: build_initial_monmap 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.422+0000 7fc782485640 10 monclient: monmap: 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:37.423314+0000 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:37.423314+0000 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:37.425 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:37.426 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:37.426 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:37.426 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 1 Processor -- start 2026-02-22T23:27:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 1 -- start start 2026-02-22T23:27:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient: init 2026-02-22T23:27:37.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:37.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient: _add_conns ranks=[2,4,1,0,5,3] 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc77c06ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc77c0c99a0 0x7fc77c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient(hunting): picked mon.noname-b con 0x7fc77c0c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc77c15a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c0c7e90 con 0x7fc77c15a5e0 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c0c8170 con 0x7fc77c06ff20 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc77c159a40 con 0x7fc77c0c99a0 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.423+0000 7fc782485640 10 monclient(hunting): authenticate will time out at 1332.150661s 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77bfff640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fc77c06ff20 0x7fc77c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:41518/0 (socket says 10.20.193.6:41518) 2026-02-22T23:27:37.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77bfff640 1 -- 10.20.193.6:0/506508048 learned_addr learned my addr 10.20.193.6:0/506508048 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 1 --2- 10.20.193.6:0/506508048 >> v2:10.20.193.40:3300/0 conn(0x7fc77c0c99a0 0x7fc77c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 10 monclient(hunting): get_auth_request con 0x7fc77c0c99a0 auth_method 0 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.424+0000 7fc77b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.425+0000 7fc77affd640 1 -- 10.20.193.6:0/506508048 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3935423800 0 0) 0x7fc77c0c8170 con 0x7fc77c06ff20 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.425+0000 7fc77affd640 10 monclient(hunting): _negotiate 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.425+0000 7fc77affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.425+0000 7fc77affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:37.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.425+0000 7fc77affd640 10 monclient(hunting): my global_id is 5105 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.425+0000 7fc77affd640 1 -- 10.20.193.6:0/506508048 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc760003620 con 0x7fc77c06ff20 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 10 monclient(hunting): handle_auth_done global_id 4978 payload 274 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 1 -- 10.20.193.6:0/506508048 >> v1:10.20.193.80:6789/0 conn(0x7fc77c06ff20 legacy=0x7fc77c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 1 -- 10.20.193.6:0/506508048 >> v1:10.20.193.40:6789/0 conn(0x7fc77c15a5e0 legacy=0x7fc77c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 1 monclient: found mon.noname-b 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:27:37.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.426+0000 7fc77b7fe640 1 -- 10.20.193.6:0/506508048 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c0c8450 con 0x7fc77c0c99a0 2026-02-22T23:27:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.428+0000 7fc77b7fe640 1 --2- 10.20.193.6:0/506508048 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c0c99a0 0x7fc77c070300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098d0 tx=0x7fc76c02eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f9c2b2b6d3f229 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 1 -- 10.20.193.6:0/506508048 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c03c070 con 0x7fc77c0c99a0 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: dump: 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: _finish_auth 0 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 1 -- 10.20.193.6:0/506508048 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76c02fba0 con 0x7fc77c0c99a0 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 5 monclient: authenticate success, global_id 4978 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 1 -- 10.20.193.6:0/506508048 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c037440 con 0x7fc77c0c99a0 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 4 set_mon_vals no callback set 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc77affd640 10 monclient: dump: 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 10 monclient: shutdown 2026-02-22T23:27:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 1 -- 10.20.193.6:0/506508048 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c0c99a0 msgr2=0x7fc77c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 1 --2- 10.20.193.6:0/506508048 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c0c99a0 0x7fc77c070300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0098d0 tx=0x7fc76c02eed0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 1 -- 10.20.193.6:0/506508048 shutdown_connections 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 1 --2- 10.20.193.6:0/506508048 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c0c99a0 0x7fc77c070300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 1 -- 10.20.193.6:0/506508048 >> 10.20.193.6:0/506508048 conn(0x7fc77c0cd780 msgr2=0x7fc77c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 1 -- 10.20.193.6:0/506508048 shutdown_connections 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 1 -- 10.20.193.6:0/506508048 wait complete. 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 10 monclient: build_initial_monmap 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.430+0000 7fc782485640 10 monclient: monmap: 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:37.440 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 librados: starting msgr at 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 librados: starting objecter 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 Processor -- start 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 -- start start 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 librados: setting wanted keys 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 librados: calling monclient init 2026-02-22T23:27:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient: init 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc77c06ff20 0x7fc77c1ed490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): picked mon.noname-a con 0x7fc77c06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc77c0c99a0 0x7fc77c0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): picked mon.noname-c con 0x7fc77c0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c15a5e0 0x7fc77c06b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): picked mon.noname-b con 0x7fc77c15a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c150520 con 0x7fc77c06ff20 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c0c8a10 con 0x7fc77c15a5e0 2026-02-22T23:27:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c1506a0 con 0x7fc77c0c99a0 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.431+0000 7fc782485640 10 monclient(hunting): authenticate will time out at 1332.158665s 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c15a5e0 0x7fc77c06b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc77c06ff20 0x7fc77c1ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc77c0c99a0 0x7fc77c0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c15a5e0 0x7fc77c06b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:53592/0 (socket says 10.20.193.6:53592) 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 1 -- 10.20.193.6:0/2726843435 learned_addr learned my addr 10.20.193.6:0/2726843435 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 10 monclient(hunting): get_auth_request con 0x7fc77c15a5e0 auth_method 0 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 10 monclient(hunting): get_auth_request con 0x7fc77c0c99a0 auth_method 0 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 10 monclient(hunting): get_auth_request con 0x7fc77c06ff20 auth_method 0 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc7809fb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.432+0000 7fc77bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 10 monclient(hunting): handle_auth_done global_id 4984 payload 824 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 1 -- 10.20.193.6:0/2726843435 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc77c0c99a0 msgr2=0x7fc77c0c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc77c0c99a0 0x7fc77c0c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 1 -- 10.20.193.6:0/2726843435 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc77c06ff20 msgr2=0x7fc77c1ed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc77c06ff20 0x7fc77c1ed490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 10 monclient: _finish_hunting 0 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 1 monclient: found mon.noname-b 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:27:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.433+0000 7fc7809fb640 1 -- 10.20.193.6:0/2726843435 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c1f37e0 con 0x7fc77c15a5e0 2026-02-22T23:27:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.434+0000 7fc7809fb640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c15a5e0 0x7fc77c06b620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cbd0 tx=0x7fc77000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.434+0000 7fc778ff9640 10 client.?.objecter ms_handle_connect 0x7fc77c15a5e0 2026-02-22T23:27:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.434+0000 7fc778ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:37.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 1 -- 10.20.193.6:0/2726843435 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77000ad50 con 0x7fc77c15a5e0 2026-02-22T23:27:37.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: dump: 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:37.448 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: _finish_auth 0 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 1 -- 10.20.193.6:0/2726843435 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770003a70 con 0x7fc77c15a5e0 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 1 -- 10.20.193.6:0/2726843435 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770003e40 con 0x7fc77c15a5e0 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc778ff9640 10 monclient: dump: 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:37.449 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 5 monclient: authenticate success, global_id 4984 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc77a7fc640 4 set_mon_vals no callback set 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 10 monclient: _renew_subs 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c0c24a0 con 0x7fc77c15a5e0 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 10 client.4984.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 10 monclient: _renew_subs 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c0c2a60 con 0x7fc77c15a5e0 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.436+0000 7fc782485640 1 librados: init done 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.437+0000 7fc782485640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.437+0000 7fc782485640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.437+0000 7fc782485640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.437+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c070300 con 0x7fc77c15a5e0 2026-02-22T23:27:37.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.541+0000 7fc778ff9640 1 -- 10.20.193.6:0/2726843435 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc7700050d0 con 0x7fc77c15a5e0 2026-02-22T23:27:37.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.542+0000 7fc778ff9640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc74c078e00 0x7fc74c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:37.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.542+0000 7fc77bfff640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc74c078e00 0x7fc74c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:37.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.542+0000 7fc77bfff640 10 monclient: get_auth_request con 0x7fc74c078e00 auth_method 0 2026-02-22T23:27:37.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.542+0000 7fc778ff9640 10 client.4984.objecter ms_handle_connect 0x7fc74c078e00 2026-02-22T23:27:37.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.542+0000 7fc77bfff640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc74c078e00 0x7fc74c07b2c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc768005980 tx=0x7fc768007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:37.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.543+0000 7fc778ff9640 1 -- 10.20.193.6:0/2726843435 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc77005e6b0 con 0x7fc77c15a5e0 2026-02-22T23:27:37.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.543+0000 7fc778ff9640 10 client.4984.objecter ms_dispatch2 0x7fc77c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:37.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.543+0000 7fc778ff9640 3 client.4984.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:37.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.543+0000 7fc778ff9640 3 client.4984.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:37.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.543+0000 7fc778ff9640 20 client.4984.objecter dump_active .. 0 homeless 2026-02-22T23:27:37.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.549+0000 7fc778ff9640 1 -- 10.20.193.6:0/2726843435 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc77c1f37e0 con 0x7fc77c15a5e0 2026-02-22T23:27:37.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.549+0000 7fc778ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:37.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.549+0000 7fc778ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:37.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.649+0000 7fc782485640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:37.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.649+0000 7fc782485640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:37.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.649+0000 7fc782485640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:27:37.651 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.649+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc77c14f380 con 0x7fc77c15a5e0 2026-02-22T23:27:37.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc778ff9640 1 -- 10.20.193.6:0/2726843435 <== mon.1 v2:10.20.193.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc770062670 con 0x7fc77c15a5e0 2026-02-22T23:27:37.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc778ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:37.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc778ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:37.653 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:37.653 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:37.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 10 librados: watch_flush enter 2026-02-22T23:27:37.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 10 librados: watch_flush exit 2026-02-22T23:27:37.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 20 client.4984.objecter shutdown clearing up homeless session... 2026-02-22T23:27:37.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 10 client.4984.objecter successfully canceled tick 2026-02-22T23:27:37.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc74c078e00 msgr2=0x7fc74c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:37.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc74c078e00 0x7fc74c07b2c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc768005980 tx=0x7fc768007dd0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 10 monclient: shutdown 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c15a5e0 msgr2=0x7fc77c06b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c15a5e0 0x7fc77c06b620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cbd0 tx=0x7fc77000a430 comp rx=0 tx=0).stop 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 shutdown_connections 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc74c078e00 0x7fc74c07b2c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc77c15a5e0 0x7fc77c06b620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc77c0c99a0 0x7fc77c0c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 --2- 10.20.193.6:0/2726843435 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc77c06ff20 0x7fc77c1ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 >> 10.20.193.6:0/2726843435 conn(0x7fc77c0cd780 msgr2=0x7fc77c072480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 shutdown_connections 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 -- 10.20.193.6:0/2726843435 wait complete. 2026-02-22T23:27:37.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:37.652+0000 7fc782485640 1 librados: shutdown 2026-02-22T23:27:40.663 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:40.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.731+0000 7ff58dfd3640 10 monclient: get_monmap_and_config 2026-02-22T23:27:40.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.731+0000 7ff58dfd3640 10 monclient: build_initial_monmap 2026-02-22T23:27:40.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.731+0000 7ff58dfd3640 10 monclient: monmap: 2026-02-22T23:27:40.733 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:40.732253+0000 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:40.732253+0000 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:40.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 1 Processor -- start 2026-02-22T23:27:40.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 1 -- start start 2026-02-22T23:27:40.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient: init 2026-02-22T23:27:40.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:40.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient: _add_conns ranks=[5,1,4,2,3,0] 2026-02-22T23:27:40.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff5881527d0 0x7ff58806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:40.737 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient(hunting): picked mon.noname-c con 0x7ff5881527d0 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7ff588152bb0 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7ff5880c6190 0x7ff5880c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient(hunting): picked mon.noname-b con 0x7ff5880c6190 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58806e300 con 0x7ff588152bb0 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff58806f1f0 con 0x7ff5880c6190 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff58806e5e0 con 0x7ff5881527d0 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.732+0000 7ff58dfd3640 10 monclient(hunting): authenticate will time out at 1335.459875s 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff587fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7ff5880c6190 0x7ff5880c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff5877fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff5881527d0 0x7ff58806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:40.738 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff587fff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7ff5880c6190 0x7ff5880c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:51396/0 (socket says 10.20.193.6:51396) 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff587fff640 1 -- 10.20.193.6:0/1711131858 learned_addr learned my addr 10.20.193.6:0/1711131858 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff587fff640 10 monclient(hunting): get_auth_request con 0x7ff5880c6190 auth_method 0 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff587fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff587fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff587fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff5877fe640 10 monclient(hunting): get_auth_request con 0x7ff5881527d0 auth_method 0 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff5877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff5877fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.733+0000 7ff5877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.734+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.734+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.734+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:40.739 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.734+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.734+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.734+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 10 monclient(hunting): handle_auth_done global_id 4987 payload 274 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 1 -- 10.20.193.6:0/1711131858 >> v2:10.20.193.80:3300/0 conn(0x7ff5881527d0 msgr2=0x7ff58806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 1 --2- 10.20.193.6:0/1711131858 >> v2:10.20.193.80:3300/0 conn(0x7ff5881527d0 0x7ff58806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 1 -- 10.20.193.6:0/1711131858 >> v1:10.20.193.40:6789/0 conn(0x7ff588152bb0 legacy=0x7ff5880c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 10 monclient: _finish_hunting 0 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 1 monclient: found mon.noname-b 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:27:40.740 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.735+0000 7ff587fff640 1 -- 10.20.193.6:0/1711131858 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58806e8c0 con 0x7ff5880c6190 2026-02-22T23:27:40.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.736+0000 7ff587fff640 1 --2- 10.20.193.6:0/1711131858 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5880c6190 0x7ff5880c8e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff5880caec0 tx=0x7ff57c02f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3477e6ca85c30f27 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:40.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff5867fc640 1 -- 10.20.193.6:0/1711131858 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c02fd70 con 0x7ff5880c6190 2026-02-22T23:27:40.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff5867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:40.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff5867fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:27:40.747 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff5867fc640 10 monclient: dump: 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff5867fc640 10 monclient: _finish_auth 0 2026-02-22T23:27:40.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff5867fc640 10 monclient: _check_auth_tickets 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff5867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff58dfd3640 5 monclient: authenticate success, global_id 4987 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.737+0000 7ff58dfd3640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff5867fc640 1 -- 10.20.193.6:0/1711131858 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c0373f0 con 0x7ff5880c6190 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff5867fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff5867fc640 1 -- 10.20.193.6:0/1711131858 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0377c0 con 0x7ff5880c6190 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff5867fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff5867fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff5867fc640 10 monclient: dump: 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:40.749 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 4 set_mon_vals no callback set 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 10 monclient: shutdown 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 -- 10.20.193.6:0/1711131858 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5880c6190 msgr2=0x7ff5880c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:40.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/1711131858 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5880c6190 0x7ff5880c8e60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff5880caec0 tx=0x7ff57c02f270 comp rx=0 tx=0).stop 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 -- 10.20.193.6:0/1711131858 shutdown_connections 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/1711131858 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff5880c6190 0x7ff5880c8e60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/1711131858 >> v2:10.20.193.80:3300/0 conn(0x7ff5881527d0 0x7ff58806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 -- 10.20.193.6:0/1711131858 >> 10.20.193.6:0/1711131858 conn(0x7ff5880cd780 msgr2=0x7ff5880cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 -- 10.20.193.6:0/1711131858 shutdown_connections 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.738+0000 7ff58dfd3640 1 -- 10.20.193.6:0/1711131858 wait complete. 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 10 monclient: build_initial_monmap 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 10 monclient: monmap: 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:40.751 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 1 librados: starting msgr at 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 1 librados: starting objecter 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 1 Processor -- start 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 1 -- start start 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 1 librados: setting wanted keys 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 1 librados: calling monclient init 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.739+0000 7ff58dfd3640 10 monclient: init 2026-02-22T23:27:40.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5880c6190 0x7ff588143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): picked mon.noname-c con 0x7ff5880c6190 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5881527d0 0x7ff588141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): picked mon.noname-a con 0x7ff5881527d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff588152bb0 0x7ff588145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): picked mon.noname-b con 0x7ff588152bb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff58806dfa0 con 0x7ff5881527d0 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff58806ee80 con 0x7ff588152bb0 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:40.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff5880c93a0 con 0x7ff5880c6190 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff58dfd3640 10 monclient(hunting): authenticate will time out at 1335.467167s 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5880c6190 0x7ff588143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff587fff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff588152bb0 0x7ff588145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff586ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5881527d0 0x7ff588141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5880c6190 0x7ff588143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:42862/0 (socket says 10.20.193.6:42862) 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 1 -- 10.20.193.6:0/2944340102 learned_addr learned my addr 10.20.193.6:0/2944340102 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 10 monclient(hunting): get_auth_request con 0x7ff5880c6190 auth_method 0 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:40.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff587fff640 10 monclient(hunting): get_auth_request con 0x7ff588152bb0 auth_method 0 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff587fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff587fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff587fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff586ffd640 10 monclient(hunting): get_auth_request con 0x7ff5881527d0 auth_method 0 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff586ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff586ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff586ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.740+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.741+0000 7ff5877fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.741+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.741+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:40.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.741+0000 7ff587fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.741+0000 7ff586ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.741+0000 7ff586ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.741+0000 7ff586ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 10 monclient(hunting): handle_auth_done global_id 5114 payload 824 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 1 -- 10.20.193.6:0/2944340102 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff588152bb0 msgr2=0x7ff588145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff588152bb0 0x7ff588145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 1 -- 10.20.193.6:0/2944340102 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5881527d0 msgr2=0x7ff588141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5881527d0 0x7ff588141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 1 monclient: found mon.noname-c 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:40.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.742+0000 7ff5877fe640 1 -- 10.20.193.6:0/2944340102 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5880bb3f0 con 0x7ff5880c6190 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.743+0000 7ff5877fe640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5880c6190 0x7ff588143f50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff57400ec10 tx=0x7ff57400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.743+0000 7ff563fff640 10 client.?.objecter ms_handle_connect 0x7ff5880c6190 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.743+0000 7ff563fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 1 -- 10.20.193.6:0/2944340102 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574017070 con 0x7ff5880c6190 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: dump: 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:40.757 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: _finish_auth 0 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: _check_auth_tickets 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 1 -- 10.20.193.6:0/2944340102 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57400cc40 con 0x7ff5880c6190 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 5 monclient: authenticate success, global_id 5114 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 monclient: _renew_subs 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:40.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5880bb6d0 con 0x7ff5880c6190 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 1 -- 10.20.193.6:0/2944340102 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574013470 con 0x7ff5880c6190 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff563fff640 10 monclient: dump: 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:40.759 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff585ffb640 4 set_mon_vals no callback set 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 client.5114.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 monclient: _renew_subs 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5880bbc90 con 0x7ff5880c6190 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 1 librados: init done 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:40.760 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.745+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5880c9730 con 0x7ff5880c6190 2026-02-22T23:27:40.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.849+0000 7ff563fff640 1 -- 10.20.193.6:0/2944340102 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff574013630 con 0x7ff5880c6190 2026-02-22T23:27:40.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.849+0000 7ff563fff640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff558078e00 0x7ff55807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:40.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.849+0000 7ff586ffd640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff558078e00 0x7ff55807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:40.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.849+0000 7ff586ffd640 10 monclient: get_auth_request con 0x7ff558078e00 auth_method 0 2026-02-22T23:27:40.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.850+0000 7ff563fff640 10 client.5114.objecter ms_handle_connect 0x7ff558078e00 2026-02-22T23:27:40.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.850+0000 7ff586ffd640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff558078e00 0x7ff55807b2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5880cc470 tx=0x7ff578007f30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:40.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.851+0000 7ff563fff640 1 -- 10.20.193.6:0/2944340102 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7ff57405e560 con 0x7ff5880c6190 2026-02-22T23:27:40.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.851+0000 7ff563fff640 10 client.5114.objecter ms_dispatch2 0x7ff588000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:40.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.851+0000 7ff563fff640 3 client.5114.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:40.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.851+0000 7ff563fff640 3 client.5114.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:40.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.851+0000 7ff563fff640 20 client.5114.objecter dump_active .. 0 homeless 2026-02-22T23:27:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.857+0000 7ff563fff640 1 -- 10.20.193.6:0/2944340102 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff574062520 con 0x7ff5880c6190 2026-02-22T23:27:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.857+0000 7ff563fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.857+0000 7ff563fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:40.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.957+0000 7ff58dfd3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.957+0000 7ff58dfd3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.957+0000 7ff58dfd3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.957+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff5881562f0 con 0x7ff5880c6190 2026-02-22T23:27:40.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff563fff640 1 -- 10.20.193.6:0/2944340102 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7ff574066820 con 0x7ff5880c6190 2026-02-22T23:27:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff563fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff563fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:40.961 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:40.961 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 10 librados: watch_flush enter 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 10 librados: watch_flush exit 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 20 client.5114.objecter shutdown clearing up homeless session... 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 10 client.5114.objecter successfully canceled tick 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff558078e00 msgr2=0x7ff55807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff558078e00 0x7ff55807b2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff5880cc470 tx=0x7ff578007f30 comp rx=0 tx=0).stop 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 10 monclient: shutdown 2026-02-22T23:27:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5880c6190 msgr2=0x7ff588143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5880c6190 0x7ff588143f50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff57400ec10 tx=0x7ff57400c520 comp rx=0 tx=0).stop 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 shutdown_connections 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff558078e00 0x7ff55807b2c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff588152bb0 0x7ff588145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff5881527d0 0x7ff588141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 --2- 10.20.193.6:0/2944340102 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff5880c6190 0x7ff588143f50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 >> 10.20.193.6:0/2944340102 conn(0x7ff5880cd780 msgr2=0x7ff5880c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 shutdown_connections 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 -- 10.20.193.6:0/2944340102 wait complete. 2026-02-22T23:27:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:40.960+0000 7ff58dfd3640 1 librados: shutdown 2026-02-22T23:27:43.971 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:44.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.037+0000 7f5064bc3640 10 monclient: get_monmap_and_config 2026-02-22T23:27:44.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.037+0000 7f5064bc3640 10 monclient: build_initial_monmap 2026-02-22T23:27:44.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.037+0000 7f5064bc3640 10 monclient: monmap: 2026-02-22T23:27:44.039 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:44.039 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:44.039 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:44.038305+0000 2026-02-22T23:27:44.039 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:44.038305+0000 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:44.040 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:44.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 1 Processor -- start 2026-02-22T23:27:44.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 1 -- start start 2026-02-22T23:27:44.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient: init 2026-02-22T23:27:44.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient: _add_conns ranks=[2,4,3,0,5,1] 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f506006ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f50600c99a0 0x7f5060070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient(hunting): picked mon.noname-b con 0x7f50600c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f506015a5e0 0x7f506014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient(hunting): picked mon.noname-a con 0x7f506015a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50600c7e90 con 0x7f506006ff20 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f5060159a40 con 0x7f506015a5e0 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f50600c8170 con 0x7f50600c99a0 2026-02-22T23:27:44.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.038+0000 7f5064bc3640 10 monclient(hunting): authenticate will time out at 1338.765776s 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505dd74640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f50600c99a0 0x7f5060070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505e575640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f506006ff20 0x7f5060159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:59926/0 (socket says 10.20.193.6:59926) 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505e575640 1 -- 10.20.193.6:0/1369385454 learned_addr learned my addr 10.20.193.6:0/1369385454 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 1 --2- 10.20.193.6:0/1369385454 >> v2:10.20.193.6:3300/0 conn(0x7f506015a5e0 0x7f506014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 10 monclient(hunting): get_auth_request con 0x7f506015a5e0 auth_method 0 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505dd74640 10 monclient(hunting): get_auth_request con 0x7f50600c99a0 auth_method 0 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:44.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505dd74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.039+0000 7f505dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 10 monclient(hunting): handle_auth_done global_id 5031 payload 274 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 1 -- 10.20.193.6:0/1369385454 >> v2:10.20.193.40:3300/0 conn(0x7f50600c99a0 msgr2=0x7f5060070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 1 --2- 10.20.193.6:0/1369385454 >> v2:10.20.193.40:3300/0 conn(0x7f50600c99a0 0x7f5060070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 1 -- 10.20.193.6:0/1369385454 >> v1:10.20.193.80:6789/0 conn(0x7f506006ff20 legacy=0x7f5060159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 10 monclient: _finish_hunting 0 2026-02-22T23:27:44.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 1 monclient: found mon.noname-a 2026-02-22T23:27:44.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:44.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:44.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505ed76640 1 -- 10.20.193.6:0/1369385454 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50600c8450 con 0x7f506015a5e0 2026-02-22T23:27:44.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505d573640 1 -- 10.20.193.6:0/1369385454 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269832146 0 0) 0x7f50600c7e90 con 0x7f506006ff20 2026-02-22T23:27:44.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.040+0000 7f505d573640 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) 2026-02-22T23:27:44.047 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.041+0000 7f505ed76640 1 --2- 10.20.193.6:0/1369385454 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506015a5e0 0x7f506014fe10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5050009d90 tx=0x7f505002f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e83fb3e7bb84e94 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:44.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 1 -- 10.20.193.6:0/1369385454 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505003c070 con 0x7f506015a5e0 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: dump: 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:44.053 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: _finish_auth 0 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: _check_auth_tickets 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 1 -- 10.20.193.6:0/1369385454 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5050037440 con 0x7f506015a5e0 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 5 monclient: authenticate success, global_id 5031 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 1 -- 10.20.193.6:0/1369385454 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050037810 con 0x7f506015a5e0 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:44.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f505d573640 10 monclient: dump: 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:44.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 4 set_mon_vals no callback set 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 10 monclient: shutdown 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 1 -- 10.20.193.6:0/1369385454 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506015a5e0 msgr2=0x7f506014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 1 --2- 10.20.193.6:0/1369385454 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506015a5e0 0x7f506014fe10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5050009d90 tx=0x7f505002f1e0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 1 -- 10.20.193.6:0/1369385454 shutdown_connections 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 1 --2- 10.20.193.6:0/1369385454 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506015a5e0 0x7f506014fe10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.045+0000 7f5064bc3640 1 --2- 10.20.193.6:0/1369385454 >> v2:10.20.193.40:3300/0 conn(0x7f50600c99a0 0x7f5060070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 -- 10.20.193.6:0/1369385454 >> 10.20.193.6:0/1369385454 conn(0x7f50600cd780 msgr2=0x7f50600cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 -- 10.20.193.6:0/1369385454 shutdown_connections 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 -- 10.20.193.6:0/1369385454 wait complete. 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 10 monclient: build_initial_monmap 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 10 monclient: monmap: 2026-02-22T23:27:44.056 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 librados: starting msgr at 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 librados: starting objecter 2026-02-22T23:27:44.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 Processor -- start 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 -- start start 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 librados: setting wanted keys 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 1 librados: calling monclient init 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.046+0000 7f5064bc3640 10 monclient: init 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506006ff20 0x7f50600c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): picked mon.noname-a con 0x7f506006ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50600c99a0 0x7f5060159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): picked mon.noname-c con 0x7f50600c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f506015a5e0 0x7f506006e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:44.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): picked mon.noname-b con 0x7f506015a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5060150350 con 0x7f506006ff20 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f50600c8a10 con 0x7f506015a5e0 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f50601504d0 con 0x7f50600c99a0 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f5064bc3640 10 monclient(hunting): authenticate will time out at 1338.774366s 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506006ff20 0x7f50600c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506006ff20 0x7f50600c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:40506/0 (socket says 10.20.193.6:40506) 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 1 -- 10.20.193.6:0/4192568318 learned_addr learned my addr 10.20.193.6:0/4192568318 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 10 monclient(hunting): get_auth_request con 0x7f506006ff20 auth_method 0 2026-02-22T23:27:44.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505dd74640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50600c99a0 0x7f5060159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505ed76640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f506015a5e0 0x7f506006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.047+0000 7f505e575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505e575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505dd74640 10 monclient(hunting): get_auth_request con 0x7f50600c99a0 auth_method 0 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505dd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505dd74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505dd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:44.060 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505ed76640 10 monclient(hunting): get_auth_request con 0x7f506015a5e0 auth_method 0 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505ed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505ed76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505ed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505dd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505dd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505dd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505ed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505ed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.048+0000 7f505ed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 10 monclient(hunting): handle_auth_done global_id 5034 payload 824 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 1 -- 10.20.193.6:0/4192568318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50600c99a0 msgr2=0x7f5060159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50600c99a0 0x7f5060159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 1 -- 10.20.193.6:0/4192568318 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f506015a5e0 msgr2=0x7f506006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f506015a5e0 0x7f506006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 10 monclient: _finish_hunting 0 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 1 monclient: found mon.noname-a 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.049+0000 7f505e575640 1 -- 10.20.193.6:0/4192568318 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50600c20a0 con 0x7f506006ff20 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.050+0000 7f505e575640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506006ff20 0x7f50600c3d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f505400c700 tx=0x7f505400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.050+0000 7f5046ffd640 10 client.?.objecter ms_handle_connect 0x7f506006ff20 2026-02-22T23:27:44.062 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.050+0000 7f5046ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:44.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 1 -- 10.20.193.6:0/4192568318 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054018070 con 0x7f506006ff20 2026-02-22T23:27:44.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:44.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:44.064 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: dump: 2026-02-22T23:27:44.064 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:44.064 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: _finish_auth 0 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:44.065 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 1 -- 10.20.193.6:0/4192568318 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5054004ae0 con 0x7f506006ff20 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 5 monclient: authenticate success, global_id 5034 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 1 -- 10.20.193.6:0/4192568318 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50540070d0 con 0x7f506006ff20 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 monclient: _renew_subs 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f505cd72640 4 set_mon_vals no callback set 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50600c2380 con 0x7f506006ff20 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5046ffd640 10 monclient: dump: 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:44.066 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 client.5034.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 monclient: _renew_subs 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:44.067 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50601f4800 con 0x7f506006ff20 2026-02-22T23:27:44.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 1 librados: init done 2026-02-22T23:27:44.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:44.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:44.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:44.068 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.052+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50601f45f0 con 0x7f506006ff20 2026-02-22T23:27:44.069 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.063+0000 7f5046ffd640 1 -- 10.20.193.6:0/4192568318 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5054007270 con 0x7f506006ff20 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.063+0000 7f5046ffd640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f502c078e80 0x7f502c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.064+0000 7f505dd74640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f502c078e80 0x7f502c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.064+0000 7f505dd74640 10 monclient: get_auth_request con 0x7f502c078e80 auth_method 0 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.064+0000 7f505dd74640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f502c078e80 0x7f502c07b340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f506014c470 tx=0x7f5048009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.064+0000 7f5046ffd640 10 client.5034.objecter ms_handle_connect 0x7f502c078e80 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.065+0000 7f5046ffd640 1 -- 10.20.193.6:0/4192568318 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f505405d4e0 con 0x7f506006ff20 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.065+0000 7f5046ffd640 10 client.5034.objecter ms_dispatch2 0x7f5060000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.065+0000 7f5046ffd640 3 client.5034.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.065+0000 7f5046ffd640 3 client.5034.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:44.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.065+0000 7f5046ffd640 20 client.5034.objecter dump_active .. 0 homeless 2026-02-22T23:27:44.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.164+0000 7f5046ffd640 1 -- 10.20.193.6:0/4192568318 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5054022020 con 0x7f506006ff20 2026-02-22T23:27:44.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.164+0000 7f5046ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:44.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.164+0000 7f5046ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:44.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.263+0000 7f5064bc3640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:44.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.263+0000 7f5064bc3640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:44.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.263+0000 7f5064bc3640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:44.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.263+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5060150860 con 0x7f506006ff20 2026-02-22T23:27:44.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.266+0000 7f5046ffd640 1 -- 10.20.193.6:0/4192568318 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f50540614a0 con 0x7f506006ff20 2026-02-22T23:27:44.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.266+0000 7f5046ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:44.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.266+0000 7f5046ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:44.268 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:44.268 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:44.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.267+0000 7f5064bc3640 10 librados: watch_flush enter 2026-02-22T23:27:44.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.267+0000 7f5064bc3640 10 librados: watch_flush exit 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.267+0000 7f5064bc3640 20 client.5034.objecter shutdown clearing up homeless session... 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.267+0000 7f5064bc3640 10 client.5034.objecter successfully canceled tick 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.267+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f502c078e80 msgr2=0x7f502c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.267+0000 7f5064bc3640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f502c078e80 0x7f502c07b340 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f506014c470 tx=0x7f5048009290 comp rx=0 tx=0).stop 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.267+0000 7f5064bc3640 10 monclient: shutdown 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506006ff20 msgr2=0x7f50600c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506006ff20 0x7f50600c3d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f505400c700 tx=0x7f505400cbd0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 shutdown_connections 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f502c078e80 0x7f502c07b340 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f506015a5e0 0x7f506006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f50600c99a0 0x7f5060159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 --2- 10.20.193.6:0/4192568318 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f506006ff20 0x7f50600c3d80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:44.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 >> 10.20.193.6:0/4192568318 conn(0x7f50600cd780 msgr2=0x7f506006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:44.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 shutdown_connections 2026-02-22T23:27:44.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 -- 10.20.193.6:0/4192568318 wait complete. 2026-02-22T23:27:44.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:44.268+0000 7f5064bc3640 1 librados: shutdown 2026-02-22T23:27:47.279 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:47.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.345+0000 7fc886314640 10 monclient: get_monmap_and_config 2026-02-22T23:27:47.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.345+0000 7fc886314640 10 monclient: build_initial_monmap 2026-02-22T23:27:47.347 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.345+0000 7fc886314640 10 monclient: monmap: 2026-02-22T23:27:47.347 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:47.347 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:47.346103+0000 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:47.346103+0000 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.345+0000 7fc886314640 1 Processor -- start 2026-02-22T23:27:47.348 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.345+0000 7fc886314640 1 -- start start 2026-02-22T23:27:47.349 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.345+0000 7fc886314640 10 monclient: init 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient: _add_conns ranks=[5,1,2,3,4,0] 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc88006ff20 0x7fc880070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient(hunting): picked mon.noname-c con 0x7fc88006ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc8800c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fc88015a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8800c7e90 con 0x7fc8800c99a0 2026-02-22T23:27:47.351 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8800c8170 con 0x7fc88015a5e0 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc880159a40 con 0x7fc88006ff20 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc886314640 10 monclient(hunting): authenticate will time out at 1342.073473s 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc87f7fe640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fc8800c99a0 0x7fc880159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:52088/0 (socket says 10.20.193.6:52088) 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc87f7fe640 1 -- 10.20.193.6:0/3068341490 learned_addr learned my addr 10.20.193.6:0/3068341490 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.346+0000 7fc87ffff640 1 --2- 10.20.193.6:0/3068341490 >> v2:10.20.193.80:3300/0 conn(0x7fc88006ff20 0x7fc880070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.347+0000 7fc87ffff640 10 monclient(hunting): get_auth_request con 0x7fc88006ff20 auth_method 0 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.347+0000 7fc87ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.347+0000 7fc87ffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.347+0000 7fc87ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:47.352 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.347+0000 7fc87ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.347+0000 7fc87ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.347+0000 7fc87ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.348+0000 7fc87effd640 1 -- 10.20.193.6:0/3068341490 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490666517 0 0) 0x7fc8800c7e90 con 0x7fc8800c99a0 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.348+0000 7fc87effd640 10 monclient(hunting): _negotiate 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.348+0000 7fc87effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.348+0000 7fc87effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.348+0000 7fc87effd640 10 monclient(hunting): my global_id is 4999 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.348+0000 7fc87effd640 1 -- 10.20.193.6:0/3068341490 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc864003620 con 0x7fc8800c99a0 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 10 monclient(hunting): handle_auth_done global_id 5123 payload 274 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 1 -- 10.20.193.6:0/3068341490 >> v1:10.20.193.80:6789/0 conn(0x7fc88015a5e0 legacy=0x7fc88014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 1 -- 10.20.193.6:0/3068341490 >> v1:10.20.193.40:6789/0 conn(0x7fc8800c99a0 legacy=0x7fc880159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:47.353 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 10 monclient: _finish_hunting 0 2026-02-22T23:27:47.354 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 1 monclient: found mon.noname-c 2026-02-22T23:27:47.354 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:47.354 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:47.354 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.349+0000 7fc87ffff640 1 -- 10.20.193.6:0/3068341490 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8800c8450 con 0x7fc88006ff20 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.350+0000 7fc87ffff640 1 --2- 10.20.193.6:0/3068341490 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc88006ff20 0x7fc880070300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0098d0 tx=0x7fc86c02eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d5da40ca82cd63 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 1 -- 10.20.193.6:0/3068341490 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c03c070 con 0x7fc88006ff20 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: dump: 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.359 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: _finish_auth 0 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 1 -- 10.20.193.6:0/3068341490 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc86c02fba0 con 0x7fc88006ff20 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 5 monclient: authenticate success, global_id 5123 2026-02-22T23:27:47.360 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 1 -- 10.20.193.6:0/3068341490 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c037440 con 0x7fc88006ff20 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 4 set_mon_vals no callback set 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc87effd640 10 monclient: dump: 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:47.361 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 10 monclient: shutdown 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 1 -- 10.20.193.6:0/3068341490 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc88006ff20 msgr2=0x7fc880070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 1 --2- 10.20.193.6:0/3068341490 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc88006ff20 0x7fc880070300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0098d0 tx=0x7fc86c02eed0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 1 -- 10.20.193.6:0/3068341490 shutdown_connections 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 1 --2- 10.20.193.6:0/3068341490 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc88006ff20 0x7fc880070300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 1 -- 10.20.193.6:0/3068341490 >> 10.20.193.6:0/3068341490 conn(0x7fc8800cd780 msgr2=0x7fc8800cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:47.362 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 1 -- 10.20.193.6:0/3068341490 shutdown_connections 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.352+0000 7fc886314640 1 -- 10.20.193.6:0/3068341490 wait complete. 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 10 monclient: build_initial_monmap 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 10 monclient: monmap: 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:47.363 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 1 librados: starting msgr at 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 1 librados: starting objecter 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 1 Processor -- start 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 1 -- start start 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 1 librados: setting wanted keys 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 1 librados: calling monclient init 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.353+0000 7fc886314640 10 monclient: init 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:47.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc88006ff20 0x7fc88014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): picked mon.noname-b con 0x7fc88006ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc8800c99a0 0x7fc8801553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): picked mon.noname-c con 0x7fc8800c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc88015a5e0 0x7fc88006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): picked mon.noname-a con 0x7fc88015a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc880150520 con 0x7fc88015a5e0 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc8800c8a10 con 0x7fc88006ff20 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:47.365 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc8801506a0 con 0x7fc8800c99a0 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc886314640 10 monclient(hunting): authenticate will time out at 1342.081186s 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc8800c99a0 0x7fc8801553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc88015a5e0 0x7fc88006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc8800c99a0 0x7fc8801553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:42896/0 (socket says 10.20.193.6:42896) 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 1 -- 10.20.193.6:0/2585569114 learned_addr learned my addr 10.20.193.6:0/2585569114 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 10 monclient(hunting): get_auth_request con 0x7fc8800c99a0 auth_method 0 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87ffff640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc88006ff20 0x7fc88014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:47.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 10 monclient(hunting): get_auth_request con 0x7fc88015a5e0 auth_method 0 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87ffff640 10 monclient(hunting): get_auth_request con 0x7fc88006ff20 auth_method 0 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87ffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc87ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:47.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.354+0000 7fc88488a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.355+0000 7fc87ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.355+0000 7fc87ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.355+0000 7fc87ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 10 monclient(hunting): handle_auth_done global_id 5129 payload 824 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 1 -- 10.20.193.6:0/2585569114 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc88006ff20 msgr2=0x7fc88014cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc88006ff20 0x7fc88014cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 1 -- 10.20.193.6:0/2585569114 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc88015a5e0 msgr2=0x7fc88006e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc88015a5e0 0x7fc88006e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 1 monclient: found mon.noname-c 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:27:47.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.356+0000 7fc87f7fe640 1 -- 10.20.193.6:0/2585569114 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88014d1e0 con 0x7fc8800c99a0 2026-02-22T23:27:47.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.357+0000 7fc87f7fe640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc8800c99a0 0x7fc8801553a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c730 tx=0x7fc87000cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:47.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.357+0000 7fc87cff9640 10 client.?.objecter ms_handle_connect 0x7fc8800c99a0 2026-02-22T23:27:47.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.357+0000 7fc87cff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:47.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 1 -- 10.20.193.6:0/2585569114 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870017070 con 0x7fc8800c99a0 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: dump: 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:47.371 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: _finish_auth 0 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: _check_auth_tickets 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 1 -- 10.20.193.6:0/2585569114 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc870005bf0 con 0x7fc8800c99a0 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 1 -- 10.20.193.6:0/2585569114 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870007070 con 0x7fc8800c99a0 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 5 monclient: authenticate success, global_id 5129 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 10 monclient: _renew_subs 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87e7fc640 4 set_mon_vals no callback set 2026-02-22T23:27:47.372 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc880155de0 con 0x7fc8800c99a0 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc87cff9640 10 monclient: dump: 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:47.373 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 10 client.5129.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 10 monclient: _renew_subs 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8801563a0 con 0x7fc8800c99a0 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.358+0000 7fc886314640 1 librados: init done 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.359+0000 7fc886314640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.359+0000 7fc886314640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.359+0000 7fc886314640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:47.374 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.359+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc880070300 con 0x7fc8800c99a0 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.370+0000 7fc87cff9640 1 -- 10.20.193.6:0/2585569114 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc870007210 con 0x7fc8800c99a0 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.371+0000 7fc87cff9640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc850078e00 0x7fc85007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.371+0000 7fc87ffff640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc850078e00 0x7fc85007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.371+0000 7fc87ffff640 10 monclient: get_auth_request con 0x7fc850078e00 auth_method 0 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.372+0000 7fc87cff9640 10 client.5129.objecter ms_handle_connect 0x7fc850078e00 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.372+0000 7fc87ffff640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc850078e00 0x7fc85007b2c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc86c039000 tx=0x7fc86c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.372+0000 7fc87cff9640 1 -- 10.20.193.6:0/2585569114 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc87005d4a0 con 0x7fc8800c99a0 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.372+0000 7fc87cff9640 10 client.5129.objecter ms_dispatch2 0x7fc880000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.372+0000 7fc87cff9640 3 client.5129.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:47.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.372+0000 7fc87cff9640 3 client.5129.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:47.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.372+0000 7fc87cff9640 20 client.5129.objecter dump_active .. 0 homeless 2026-02-22T23:27:47.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.470+0000 7fc87cff9640 1 -- 10.20.193.6:0/2585569114 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc870021020 con 0x7fc8800c99a0 2026-02-22T23:27:47.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.470+0000 7fc87cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:47.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.470+0000 7fc87cff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:47.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.572+0000 7fc886314640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:47.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.572+0000 7fc886314640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:47.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.572+0000 7fc886314640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:27:47.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.572+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc880159820 con 0x7fc8800c99a0 2026-02-22T23:27:47.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc87cff9640 1 -- 10.20.193.6:0/2585569114 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc870061460 con 0x7fc8800c99a0 2026-02-22T23:27:47.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc87cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:47.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc87cff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:47.576 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:47.576 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:47.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 10 librados: watch_flush enter 2026-02-22T23:27:47.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 10 librados: watch_flush exit 2026-02-22T23:27:47.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 20 client.5129.objecter shutdown clearing up homeless session... 2026-02-22T23:27:47.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 10 client.5129.objecter successfully canceled tick 2026-02-22T23:27:47.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc850078e00 msgr2=0x7fc85007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:47.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc850078e00 0x7fc85007b2c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc86c039000 tx=0x7fc86c033000 comp rx=0 tx=0).stop 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 10 monclient: shutdown 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.575+0000 7fc886314640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc8800c99a0 msgr2=0x7fc8801553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc8800c99a0 0x7fc8801553a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c730 tx=0x7fc87000cc00 comp rx=0 tx=0).stop 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 shutdown_connections 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc850078e00 0x7fc85007b2c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc88015a5e0 0x7fc88006e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc8800c99a0 0x7fc8801553a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 --2- 10.20.193.6:0/2585569114 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc88006ff20 0x7fc88014cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 >> 10.20.193.6:0/2585569114 conn(0x7fc8800cd780 msgr2=0x7fc88014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:47.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 shutdown_connections 2026-02-22T23:27:47.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 -- 10.20.193.6:0/2585569114 wait complete. 2026-02-22T23:27:47.580 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:47.576+0000 7fc886314640 1 librados: shutdown 2026-02-22T23:27:50.587 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.653+0000 7fc2bef08640 10 monclient: get_monmap_and_config 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.653+0000 7fc2bef08640 10 monclient: build_initial_monmap 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.653+0000 7fc2bef08640 10 monclient: monmap: 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:50.653863+0000 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:50.653863+0000 2026-02-22T23:27:50.655 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.653+0000 7fc2bef08640 1 Processor -- start 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.653+0000 7fc2bef08640 1 -- start start 2026-02-22T23:27:50.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.653+0000 7fc2bef08640 10 monclient: init 2026-02-22T23:27:50.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:50.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient: _add_conns ranks=[3,5,4,0,2,1] 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc2b806ff20 0x7fc2b8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): picked mon.noname-a con 0x7fc2b806ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc2b80c99a0 0x7fc2b8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): picked mon.noname-c con 0x7fc2b80c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc2b815a410 0x7fc2b814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): picked mon.noname-b con 0x7fc2b815a410 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b8159a40 con 0x7fc2b806ff20 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b815a860 con 0x7fc2b815a410 2026-02-22T23:27:50.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b80c7e90 con 0x7fc2b80c99a0 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bef08640 10 monclient(hunting): authenticate will time out at 1345.381325s 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2affff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc2b80c99a0 0x7fc2b8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bcc7d640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc2b806ff20 0x7fc2b8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2bd47e640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fc2b815a410 0x7fc2b814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2affff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc2b80c99a0 0x7fc2b8159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:49804/0 (socket says 10.20.193.6:49804) 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2affff640 1 -- 10.20.193.6:0/2760825712 learned_addr learned my addr 10.20.193.6:0/2760825712 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2affff640 10 monclient(hunting): get_auth_request con 0x7fc2b80c99a0 auth_method 0 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2affff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:50.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.654+0000 7fc2affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bcc7d640 10 monclient(hunting): get_auth_request con 0x7fc2b806ff20 auth_method 0 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bcc7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bcc7d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bcc7d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bd47e640 10 monclient(hunting): get_auth_request con 0x7fc2b815a410 auth_method 0 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bd47e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bd47e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bd47e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:50.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bd47e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bd47e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.655+0000 7fc2bd47e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_done global_id 5040 payload 274 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 1 -- 10.20.193.6:0/2760825712 >> v2:10.20.193.80:3300/0 conn(0x7fc2b80c99a0 msgr2=0x7fc2b8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 1 --2- 10.20.193.6:0/2760825712 >> v2:10.20.193.80:3300/0 conn(0x7fc2b80c99a0 0x7fc2b8159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 1 -- 10.20.193.6:0/2760825712 >> v2:10.20.193.40:3300/0 conn(0x7fc2b815a410 msgr2=0x7fc2b814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 1 --2- 10.20.193.6:0/2760825712 >> v2:10.20.193.40:3300/0 conn(0x7fc2b815a410 0x7fc2b814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 10 monclient: _finish_hunting 0 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 1 monclient: found mon.noname-a 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:50.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:50.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.656+0000 7fc2bcc7d640 1 -- 10.20.193.6:0/2760825712 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b80c8170 con 0x7fc2b806ff20 2026-02-22T23:27:50.664 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.657+0000 7fc2bcc7d640 1 --2- 10.20.193.6:0/2760825712 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b8070300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80c5380 tx=0x7fc2a802ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8ba1f0403b4e50 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:50.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 1 -- 10.20.193.6:0/2760825712 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a803c070 con 0x7fc2b806ff20 2026-02-22T23:27:50.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:50.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: dump: 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:50.671 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: _finish_auth 0 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 1 -- 10.20.193.6:0/2760825712 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a802fa60 con 0x7fc2b806ff20 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 5 monclient: authenticate success, global_id 5040 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 1 -- 10.20.193.6:0/2760825712 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a802fe30 con 0x7fc2b806ff20 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2af7fe640 10 monclient: dump: 2026-02-22T23:27:50.672 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:50.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 4 set_mon_vals no callback set 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 10 monclient: shutdown 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 -- 10.20.193.6:0/2760825712 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 msgr2=0x7fc2b8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 --2- 10.20.193.6:0/2760825712 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b8070300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80c5380 tx=0x7fc2a802ede0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 -- 10.20.193.6:0/2760825712 shutdown_connections 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 --2- 10.20.193.6:0/2760825712 >> v2:10.20.193.40:3300/0 conn(0x7fc2b815a410 0x7fc2b814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 --2- 10.20.193.6:0/2760825712 >> v2:10.20.193.80:3300/0 conn(0x7fc2b80c99a0 0x7fc2b8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 --2- 10.20.193.6:0/2760825712 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b8070300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 -- 10.20.193.6:0/2760825712 >> 10.20.193.6:0/2760825712 conn(0x7fc2b80cd780 msgr2=0x7fc2b80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 -- 10.20.193.6:0/2760825712 shutdown_connections 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.659+0000 7fc2bef08640 1 -- 10.20.193.6:0/2760825712 wait complete. 2026-02-22T23:27:50.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient: build_initial_monmap 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient: monmap: 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:50.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 librados: starting msgr at 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 librados: starting objecter 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 Processor -- start 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 -- start start 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 librados: setting wanted keys 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 librados: calling monclient init 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient: init 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b80c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): picked mon.noname-a con 0x7fc2b806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2b80c99a0 0x7fc2b814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): picked mon.noname-b con 0x7fc2b80c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:50.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc2b815a410 0x7fc2b80c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): picked mon.noname-c con 0x7fc2b815a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b80c8a10 con 0x7fc2b806ff20 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b80c8730 con 0x7fc2b80c99a0 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b8150350 con 0x7fc2b815a410 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bef08640 10 monclient(hunting): authenticate will time out at 1345.387725s 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.660+0000 7fc2bcc7d640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b80c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2b80c99a0 0x7fc2b814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:50.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc2b815a410 0x7fc2b80c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b80c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:55924/0 (socket says 10.20.193.6:55924) 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 1 -- 10.20.193.6:0/276770631 learned_addr learned my addr 10.20.193.6:0/276770631 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 10 monclient(hunting): get_auth_request con 0x7fc2b806ff20 auth_method 0 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 10 monclient(hunting): get_auth_request con 0x7fc2b815a410 auth_method 0 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 10 monclient(hunting): get_auth_request con 0x7fc2b80c99a0 auth_method 0 2026-02-22T23:27:50.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2bd47e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.661+0000 7fc2affff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:50.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 10 monclient(hunting): handle_auth_done global_id 5043 payload 824 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 1 -- 10.20.193.6:0/276770631 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc2b815a410 msgr2=0x7fc2b80c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc2b815a410 0x7fc2b80c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 1 -- 10.20.193.6:0/276770631 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2b80c99a0 msgr2=0x7fc2b814b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2b80c99a0 0x7fc2b814b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 10 monclient: _finish_hunting 0 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 1 monclient: found mon.noname-a 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.662+0000 7fc2bcc7d640 1 -- 10.20.193.6:0/276770631 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b80c2240 con 0x7fc2b806ff20 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.663+0000 7fc2bcc7d640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b80c1ae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc2a803a040 tx=0x7fc2a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.663+0000 7fc2ad7fa640 10 client.?.objecter ms_handle_connect 0x7fc2b806ff20 2026-02-22T23:27:50.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.663+0000 7fc2ad7fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 1 -- 10.20.193.6:0/276770631 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a803c040 con 0x7fc2b806ff20 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: dump: 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:50.681 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: _finish_auth 0 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: _check_auth_tickets 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 5 monclient: authenticate success, global_id 5043 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 1 -- 10.20.193.6:0/276770631 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a8042430 con 0x7fc2b806ff20 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 monclient: _renew_subs 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b806d2a0 con 0x7fc2b806ff20 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 1 -- 10.20.193.6:0/276770631 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8042800 con 0x7fc2b806ff20 2026-02-22T23:27:50.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2aeffd640 4 set_mon_vals no callback set 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2ad7fa640 10 monclient: dump: 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:50.683 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 client.5043.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 monclient: _renew_subs 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b806d860 con 0x7fc2b806ff20 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 1 librados: init done 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:50.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.665+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b81506e0 con 0x7fc2b806ff20 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.676+0000 7fc2ad7fa640 1 -- 10.20.193.6:0/276770631 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc2a80429a0 con 0x7fc2b806ff20 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.677+0000 7fc2ad7fa640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc288078e50 0x7fc28807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.677+0000 7fc2affff640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc288078e50 0x7fc28807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.677+0000 7fc2affff640 10 monclient: get_auth_request con 0x7fc288078e50 auth_method 0 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.678+0000 7fc2ad7fa640 1 -- 10.20.193.6:0/276770631 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc2a8083720 con 0x7fc2b806ff20 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.678+0000 7fc2ad7fa640 10 client.5043.objecter ms_dispatch2 0x7fc2b8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.678+0000 7fc2ad7fa640 3 client.5043.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.678+0000 7fc2ad7fa640 3 client.5043.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.678+0000 7fc2affff640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc288078e50 0x7fc28807b310 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc2b806ed50 tx=0x7fc2a0009340 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:50.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.678+0000 7fc2ad7fa640 20 client.5043.objecter dump_active .. 0 homeless 2026-02-22T23:27:50.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.678+0000 7fc2ad7fa640 10 client.5043.objecter ms_handle_connect 0x7fc288078e50 2026-02-22T23:27:50.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.775+0000 7fc2ad7fa640 1 -- 10.20.193.6:0/276770631 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc2a80876e0 con 0x7fc2b806ff20 2026-02-22T23:27:50.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.775+0000 7fc2ad7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:50.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.775+0000 7fc2ad7fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:50.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.877+0000 7fc2bef08640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:50.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.877+0000 7fc2bef08640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:50.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.877+0000 7fc2bef08640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:50.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.877+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc2b8156f50 con 0x7fc2b806ff20 2026-02-22T23:27:50.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.904+0000 7fc2ad7fa640 1 -- 10.20.193.6:0/276770631 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc2a808b9e0 con 0x7fc2b806ff20 2026-02-22T23:27:50.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.904+0000 7fc2ad7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:50.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.904+0000 7fc2ad7fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:50.907 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:50.907 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:50.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 10 librados: watch_flush enter 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 10 librados: watch_flush exit 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 20 client.5043.objecter shutdown clearing up homeless session... 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 10 client.5043.objecter successfully canceled tick 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc288078e50 msgr2=0x7fc28807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc288078e50 0x7fc28807b310 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc2b806ed50 tx=0x7fc2a0009340 comp rx=0 tx=0).stop 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 10 monclient: shutdown 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:50.910 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 msgr2=0x7fc2b80c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b80c1ae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc2a803a040 tx=0x7fc2a80047c0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 shutdown_connections 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc288078e50 0x7fc28807b310 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc2b815a410 0x7fc2b80c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc2b80c99a0 0x7fc2b814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 --2- 10.20.193.6:0/276770631 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc2b806ff20 0x7fc2b80c1ae0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 >> 10.20.193.6:0/276770631 conn(0x7fc2b80cd780 msgr2=0x7fc2b814eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 shutdown_connections 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 -- 10.20.193.6:0/276770631 wait complete. 2026-02-22T23:27:50.911 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:50.906+0000 7fc2bef08640 1 librados: shutdown 2026-02-22T23:27:53.330 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:37.579 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.356 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:37.603 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.358 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:37.604 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.359 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:37.609 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.370 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:37.617 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.370 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:37.621 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.384 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:37.595 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.385 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:37.596 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.396 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:37.607 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:27:53.918 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:53.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.985+0000 7fa8c49eb640 10 monclient: get_monmap_and_config 2026-02-22T23:27:53.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.985+0000 7fa8c49eb640 10 monclient: build_initial_monmap 2026-02-22T23:27:53.988 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.985+0000 7fa8c49eb640 10 monclient: monmap: 2026-02-22T23:27:53.988 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:53.988 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:53.988 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:53.986853+0000 2026-02-22T23:27:53.988 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:53.986853+0000 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:53.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.985+0000 7fa8c49eb640 1 Processor -- start 2026-02-22T23:27:53.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.985+0000 7fa8c49eb640 1 -- start start 2026-02-22T23:27:53.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.985+0000 7fa8c49eb640 10 monclient: init 2026-02-22T23:27:53.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:53.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient: _add_conns ranks=[3,5,4,0,1,2] 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa8bc06ff20 0x7fa8bc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): picked mon.noname-a con 0x7fa8bc06ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa8bc0c99a0 0x7fa8bc159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): picked mon.noname-c con 0x7fa8bc0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa8bc15a410 0x7fa8bc14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): picked mon.noname-b con 0x7fa8bc15a410 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc159a40 con 0x7fa8bc06ff20 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc15a860 con 0x7fa8bc15a410 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:53.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc0c7e90 con 0x7fa8bc0c99a0 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c49eb640 10 monclient(hunting): authenticate will time out at 1348.714283s 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c2760640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa8bc06ff20 0x7fa8bc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c1f5f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa8bc0c99a0 0x7fa8bc159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.986+0000 7fa8c2f61640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa8bc15a410 0x7fa8bc14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa8bc0c99a0 0x7fa8bc159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:49826/0 (socket says 10.20.193.6:49826) 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 1 -- 10.20.193.6:0/3397525826 learned_addr learned my addr 10.20.193.6:0/3397525826 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 10 monclient(hunting): get_auth_request con 0x7fa8bc0c99a0 auth_method 0 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:53.998 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2760640 10 monclient(hunting): get_auth_request con 0x7fa8bc06ff20 auth_method 0 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2760640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2760640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2760640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2f61640 10 monclient(hunting): get_auth_request con 0x7fa8bc15a410 auth_method 0 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2f61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2f61640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2f61640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2760640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2760640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2760640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:53.999 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c1f5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2f61640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2f61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.987+0000 7fa8c2f61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 10 monclient(hunting): handle_auth_done global_id 5046 payload 274 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 1 -- 10.20.193.6:0/3397525826 >> v2:10.20.193.80:3300/0 conn(0x7fa8bc0c99a0 msgr2=0x7fa8bc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 1 --2- 10.20.193.6:0/3397525826 >> v2:10.20.193.80:3300/0 conn(0x7fa8bc0c99a0 0x7fa8bc159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 1 -- 10.20.193.6:0/3397525826 >> v2:10.20.193.40:3300/0 conn(0x7fa8bc15a410 msgr2=0x7fa8bc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 1 --2- 10.20.193.6:0/3397525826 >> v2:10.20.193.40:3300/0 conn(0x7fa8bc15a410 0x7fa8bc14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 10 monclient: _finish_hunting 0 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 1 monclient: found mon.noname-a 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:54.000 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.988+0000 7fa8c2760640 1 -- 10.20.193.6:0/3397525826 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc0c8170 con 0x7fa8bc06ff20 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.989+0000 7fa8c2760640 1 --2- 10.20.193.6:0/3397525826 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc06ff20 0x7fa8bc070300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0c5380 tx=0x7fa8b402ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3428ee3f05bfd403 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 1 -- 10.20.193.6:0/3397525826 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b403c070 con 0x7fa8bc06ff20 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: dump: 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:54.001 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: _finish_auth 0 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: _check_auth_tickets 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 5 monclient: authenticate success, global_id 5046 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 1 -- 10.20.193.6:0/3397525826 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b402fa60 con 0x7fa8bc06ff20 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:54.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 1 -- 10.20.193.6:0/3397525826 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b402fe30 con 0x7fa8bc06ff20 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 4 set_mon_vals no callback set 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c175e640 10 monclient: dump: 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:54.003 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 10 monclient: shutdown 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 -- 10.20.193.6:0/3397525826 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc06ff20 msgr2=0x7fa8bc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/3397525826 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc06ff20 0x7fa8bc070300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0c5380 tx=0x7fa8b402ede0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 -- 10.20.193.6:0/3397525826 shutdown_connections 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/3397525826 >> v2:10.20.193.40:3300/0 conn(0x7fa8bc15a410 0x7fa8bc14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/3397525826 >> v2:10.20.193.80:3300/0 conn(0x7fa8bc0c99a0 0x7fa8bc159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/3397525826 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc06ff20 0x7fa8bc070300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 -- 10.20.193.6:0/3397525826 >> 10.20.193.6:0/3397525826 conn(0x7fa8bc0cd780 msgr2=0x7fa8bc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:54.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 -- 10.20.193.6:0/3397525826 shutdown_connections 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.991+0000 7fa8c49eb640 1 -- 10.20.193.6:0/3397525826 wait complete. 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient: build_initial_monmap 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient: monmap: 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:54.005 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 librados: starting msgr at 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 librados: starting objecter 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 Processor -- start 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 -- start start 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 librados: setting wanted keys 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 librados: calling monclient init 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient: init 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8bc06ff20 0x7fa8bc1f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): picked mon.noname-b con 0x7fa8bc06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:54.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8bc0c99a0 0x7fa8bc152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): picked mon.noname-c con 0x7fa8bc0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc15a410 0x7fa8bc14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): picked mon.noname-a con 0x7fa8bc15a410 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0c8a10 con 0x7fa8bc15a410 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0c8730 con 0x7fa8bc06ff20 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc150350 con 0x7fa8bc0c99a0 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c49eb640 10 monclient(hunting): authenticate will time out at 1348.720756s 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.992+0000 7fa8c2f61640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc15a410 0x7fa8bc14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:54.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8bc06ff20 0x7fa8bc1f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc15a410 0x7fa8bc14ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:55940/0 (socket says 10.20.193.6:55940) 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 1 -- 10.20.193.6:0/282263419 learned_addr learned my addr 10.20.193.6:0/282263419 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 10 monclient(hunting): get_auth_request con 0x7fa8bc15a410 auth_method 0 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 10 monclient(hunting): get_auth_request con 0x7fa8bc06ff20 auth_method 0 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:54.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:54.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2f61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:54.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:54.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:54.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.993+0000 7fa8c2760640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 10 monclient(hunting): handle_auth_done global_id 5049 payload 824 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 1 -- 10.20.193.6:0/282263419 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8bc0c99a0 msgr2=0x7fa8bc152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8bc0c99a0 0x7fa8bc152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 1 -- 10.20.193.6:0/282263419 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8bc06ff20 msgr2=0x7fa8bc1f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8bc06ff20 0x7fa8bc1f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 10 monclient: _finish_hunting 0 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 1 monclient: found mon.noname-a 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:54.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.994+0000 7fa8c2f61640 1 -- 10.20.193.6:0/282263419 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc151b90 con 0x7fa8bc15a410 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.996+0000 7fa8c2f61640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc15a410 0x7fa8bc14ca00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009d30 tx=0x7fa8b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.996+0000 7fa8a6ffd640 10 client.?.objecter ms_handle_connect 0x7fa8bc15a410 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.996+0000 7fa8a6ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 1 -- 10.20.193.6:0/282263419 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0018020 con 0x7fa8bc15a410 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: dump: 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.013 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 1 -- 10.20.193.6:0/282263419 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b0014480 con 0x7fa8bc15a410 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8c49eb640 5 monclient: authenticate success, global_id 5049 2026-02-22T23:27:54.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 1 -- 10.20.193.6:0/282263419 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0014870 con 0x7fa8bc15a410 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8c0f5d640 4 set_mon_vals no callback set 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8a6ffd640 10 monclient: dump: 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:54.015 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8c49eb640 10 monclient: _renew_subs 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8c49eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.997+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc151e70 con 0x7fa8bc15a410 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 10 client.5049.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 10 monclient: _renew_subs 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc152430 con 0x7fa8bc15a410 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 1 librados: init done 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:54.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:54.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:54.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:53.998+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc1506e0 con 0x7fa8bc15a410 2026-02-22T23:27:54.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.009+0000 7fa8a6ffd640 1 -- 10.20.193.6:0/282263419 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa8b0014a10 con 0x7fa8bc15a410 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.010+0000 7fa8a6ffd640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa88c078e80 0x7fa88c07b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.010+0000 7fa8c2760640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa88c078e80 0x7fa88c07b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.010+0000 7fa8c2760640 10 monclient: get_auth_request con 0x7fa88c078e80 auth_method 0 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.011+0000 7fa8c2760640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa88c078e80 0x7fa88c07b340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4039000 tx=0x7fa8b4033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.011+0000 7fa8a6ffd640 10 client.5049.objecter ms_handle_connect 0x7fa88c078e80 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.012+0000 7fa8a6ffd640 1 -- 10.20.193.6:0/282263419 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fa8b005ee20 con 0x7fa8bc15a410 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.012+0000 7fa8a6ffd640 10 client.5049.objecter ms_dispatch2 0x7fa8bc000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.012+0000 7fa8a6ffd640 3 client.5049.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:54.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.012+0000 7fa8a6ffd640 3 client.5049.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:54.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.012+0000 7fa8a6ffd640 20 client.5049.objecter dump_active .. 0 homeless 2026-02-22T23:27:54.116 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.114+0000 7fa8a6ffd640 1 -- 10.20.193.6:0/282263419 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa8b0022030 con 0x7fa8bc15a410 2026-02-22T23:27:54.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.114+0000 7fa8a6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:54.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.114+0000 7fa8a6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:54.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.214+0000 7fa8c49eb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:54.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.214+0000 7fa8c49eb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:54.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.214+0000 7fa8c49eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:54.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.214+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa8bc074100 con 0x7fa8bc15a410 2026-02-22T23:27:54.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.216+0000 7fa8a6ffd640 1 -- 10.20.193.6:0/282263419 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fa8b0010040 con 0x7fa8bc15a410 2026-02-22T23:27:54.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.216+0000 7fa8a6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:54.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.216+0000 7fa8a6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:54.219 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:54.219 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 10 librados: watch_flush enter 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 10 librados: watch_flush exit 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 20 client.5049.objecter shutdown clearing up homeless session... 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 10 client.5049.objecter successfully canceled tick 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa88c078e80 msgr2=0x7fa88c07b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa88c078e80 0x7fa88c07b340 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4039000 tx=0x7fa8b4033000 comp rx=0 tx=0).stop 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 10 monclient: shutdown 2026-02-22T23:27:54.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc15a410 msgr2=0x7fa8bc14ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc15a410 0x7fa8bc14ca00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009d30 tx=0x7fa8b000c490 comp rx=0 tx=0).stop 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 shutdown_connections 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa88c078e80 0x7fa88c07b340 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa8bc15a410 0x7fa8bc14ca00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa8bc0c99a0 0x7fa8bc152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 --2- 10.20.193.6:0/282263419 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa8bc06ff20 0x7fa8bc1f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 >> 10.20.193.6:0/282263419 conn(0x7fa8bc0cd780 msgr2=0x7fa8bc0cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 shutdown_connections 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 -- 10.20.193.6:0/282263419 wait complete. 2026-02-22T23:27:54.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:54.217+0000 7fa8c49eb640 1 librados: shutdown 2026-02-22T23:27:57.229 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.295+0000 7fcf94355640 10 monclient: get_monmap_and_config 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.295+0000 7fcf94355640 10 monclient: build_initial_monmap 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.295+0000 7fcf94355640 10 monclient: monmap: 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:27:57.297173+0000 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:27:57.297173+0000 2026-02-22T23:27:57.301 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.295+0000 7fcf94355640 1 Processor -- start 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 1 -- start start 2026-02-22T23:27:57.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient: init 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient: _add_conns ranks=[3,2,4,1,5,0] 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcf8c06ff20 0x7fcf8c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient(hunting): picked mon.noname-a con 0x7fcf8c06ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fcf8c0c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fcf8c15a5e0 0x7fcf8c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient(hunting): picked mon.noname-b con 0x7fcf8c15a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8c0c7e90 con 0x7fcf8c0c99a0 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8c159a40 con 0x7fcf8c06ff20 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:57.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8c0c8170 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf94355640 10 monclient(hunting): authenticate will time out at 1352.024537s 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.296+0000 7fcf920ca640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcf8c06ff20 0x7fcf8c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf918c9640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fcf8c0c99a0 0x7fcf8c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:50026/0 (socket says 10.20.193.6:50026) 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf918c9640 1 -- 10.20.193.6:0/3691746483 learned_addr learned my addr 10.20.193.6:0/3691746483 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf920ca640 10 monclient(hunting): get_auth_request con 0x7fcf8c06ff20 auth_method 0 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf920ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf920ca640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf920ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:57.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 1 --2- 10.20.193.6:0/3691746483 >> v2:10.20.193.40:3300/0 conn(0x7fcf8c15a5e0 0x7fcf8c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 10 monclient(hunting): get_auth_request con 0x7fcf8c15a5e0 auth_method 0 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf920ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf920ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf920ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.297+0000 7fcf928cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 10 monclient(hunting): handle_auth_done global_id 5052 payload 274 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 1 -- 10.20.193.6:0/3691746483 >> v2:10.20.193.40:3300/0 conn(0x7fcf8c15a5e0 msgr2=0x7fcf8c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:27:57.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 1 --2- 10.20.193.6:0/3691746483 >> v2:10.20.193.40:3300/0 conn(0x7fcf8c15a5e0 0x7fcf8c14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 1 -- 10.20.193.6:0/3691746483 >> v1:10.20.193.80:6789/0 conn(0x7fcf8c0c99a0 legacy=0x7fcf8c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:57.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 10 monclient: _finish_hunting 0 2026-02-22T23:27:57.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 1 monclient: found mon.noname-a 2026-02-22T23:27:57.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:57.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:57.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.298+0000 7fcf920ca640 1 -- 10.20.193.6:0/3691746483 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c0c8450 con 0x7fcf8c06ff20 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.299+0000 7fcf920ca640 1 --2- 10.20.193.6:0/3691746483 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c06ff20 0x7fcf8c070300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0098a0 tx=0x7fcf7c02ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987e78d04c6405bc server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 1 -- 10.20.193.6:0/3691746483 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c03c070 con 0x7fcf8c06ff20 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: dump: 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: _finish_auth 0 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: _check_auth_tickets 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf94355640 5 monclient: authenticate success, global_id 5052 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf94355640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 1 -- 10.20.193.6:0/3691746483 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c02fae0 con 0x7fcf8c06ff20 2026-02-22T23:27:57.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 1 -- 10.20.193.6:0/3691746483 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c037440 con 0x7fcf8c06ff20 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf910c8640 10 monclient: dump: 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf94355640 10 monclient: get_monmap_and_config success 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.300+0000 7fcf94355640 4 set_mon_vals no callback set 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 10 monclient: shutdown 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 -- 10.20.193.6:0/3691746483 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c06ff20 msgr2=0x7fcf8c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 --2- 10.20.193.6:0/3691746483 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c06ff20 0x7fcf8c070300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0098a0 tx=0x7fcf7c02ee10 comp rx=0 tx=0).stop 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 -- 10.20.193.6:0/3691746483 shutdown_connections 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 --2- 10.20.193.6:0/3691746483 >> v2:10.20.193.40:3300/0 conn(0x7fcf8c15a5e0 0x7fcf8c14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 --2- 10.20.193.6:0/3691746483 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c06ff20 0x7fcf8c070300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 -- 10.20.193.6:0/3691746483 >> 10.20.193.6:0/3691746483 conn(0x7fcf8c0cd780 msgr2=0x7fcf8c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 -- 10.20.193.6:0/3691746483 shutdown_connections 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 -- 10.20.193.6:0/3691746483 wait complete. 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 10 monclient: build_initial_monmap 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 10 monclient: monmap: 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 librados: starting msgr at 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 librados: starting objecter 2026-02-22T23:27:57.314 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.301+0000 7fcf94355640 1 Processor -- start 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 -- start start 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 librados: setting wanted keys 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 librados: calling monclient init 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient: init 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient: _reopen_session rank -1 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcf8c06ff20 0x7fcf8c1f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): picked mon.noname-c con 0x7fcf8c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcf8c0c99a0 0x7fcf8c1f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): picked mon.noname-b con 0x7fcf8c0c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c15a5e0 0x7fcf8c14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): picked mon.noname-a con 0x7fcf8c15a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c150350 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c0c8a10 con 0x7fcf8c0c99a0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): start opening mon connection 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c1504d0 con 0x7fcf8c06ff20 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): _renew_subs 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf94355640 10 monclient(hunting): authenticate will time out at 1352.030403s 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf928cb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c15a5e0 0x7fcf8c14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf918c9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcf8c0c99a0 0x7fcf8c1f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf920ca640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcf8c06ff20 0x7fcf8c1f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf928cb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c15a5e0 0x7fcf8c14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:55964/0 (socket says 10.20.193.6:55964) 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf928cb640 1 -- 10.20.193.6:0/940463012 learned_addr learned my addr 10.20.193.6:0/940463012 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf928cb640 10 monclient(hunting): get_auth_request con 0x7fcf8c15a5e0 auth_method 0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf928cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf928cb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.302+0000 7fcf928cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf920ca640 10 monclient(hunting): get_auth_request con 0x7fcf8c06ff20 auth_method 0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf920ca640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf920ca640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf920ca640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf918c9640 10 monclient(hunting): get_auth_request con 0x7fcf8c0c99a0 auth_method 0 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf918c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf918c9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf918c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf928cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf928cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf928cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf918c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf918c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:57.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf918c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf920ca640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf920ca640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.303+0000 7fcf920ca640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 10 monclient(hunting): handle_auth_done global_id 5055 payload 824 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 1 -- 10.20.193.6:0/940463012 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcf8c06ff20 msgr2=0x7fcf8c1f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcf8c06ff20 0x7fcf8c1f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 1 -- 10.20.193.6:0/940463012 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcf8c0c99a0 msgr2=0x7fcf8c1f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcf8c0c99a0 0x7fcf8c1f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 10 monclient: _finish_hunting 0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 1 monclient: found mon.noname-a 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.304+0000 7fcf928cb640 1 -- 10.20.193.6:0/940463012 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c1f4630 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.305+0000 7fcf928cb640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c15a5e0 0x7fcf8c14c250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800efc0 tx=0x7fcf8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.305+0000 7fcf72ffd640 10 client.?.objecter ms_handle_connect 0x7fcf8c15a5e0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.305+0000 7fcf72ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 1 -- 10.20.193.6:0/940463012 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88018020 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: dump: 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: _finish_auth 0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 1 -- 10.20.193.6:0/940463012 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf88014480 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 1 -- 10.20.193.6:0/940463012 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88014870 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf908c7640 4 set_mon_vals no callback set 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf72ffd640 10 monclient: dump: 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:27:57.316 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 5 monclient: authenticate success, global_id 5055 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 10 monclient: _renew_subs 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c0c1200 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 10 client.5055.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 10 monclient: _renew_subs 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c1f4070 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.306+0000 7fcf94355640 1 librados: init done 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.307+0000 7fcf94355640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.307+0000 7fcf94355640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.307+0000 7fcf94355640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:57.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.307+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8c150860 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.319+0000 7fcf72ffd640 1 -- 10.20.193.6:0/940463012 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fcf88014a10 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.319+0000 7fcf72ffd640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf5c078e50 0x7fcf5c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:27:57.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.319+0000 7fcf920ca640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf5c078e50 0x7fcf5c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:27:57.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.319+0000 7fcf920ca640 10 monclient: get_auth_request con 0x7fcf5c078e50 auth_method 0 2026-02-22T23:27:57.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.320+0000 7fcf920ca640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf5c078e50 0x7fcf5c07b310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c039000 tx=0x7fcf7c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:27:57.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.320+0000 7fcf72ffd640 10 client.5055.objecter ms_handle_connect 0x7fcf5c078e50 2026-02-22T23:27:57.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.379+0000 7fcf72ffd640 1 -- 10.20.193.6:0/940463012 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fcf8805ee80 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.379+0000 7fcf72ffd640 10 client.5055.objecter ms_dispatch2 0x7fcf8c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:27:57.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.379+0000 7fcf72ffd640 3 client.5055.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:27:57.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.379+0000 7fcf72ffd640 3 client.5055.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:27:57.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.380+0000 7fcf72ffd640 20 client.5055.objecter dump_active .. 0 homeless 2026-02-22T23:27:57.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.414+0000 7fcf72ffd640 1 -- 10.20.193.6:0/940463012 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fcf88022030 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.414+0000 7fcf72ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:27:57.416 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.414+0000 7fcf72ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:27:57.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.514+0000 7fcf94355640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:27:57.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.514+0000 7fcf94355640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:57.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.514+0000 7fcf94355640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:27:57.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.514+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcf8c14f510 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.517+0000 7fcf72ffd640 1 -- 10.20.193.6:0/940463012 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fcf88010040 con 0x7fcf8c15a5e0 2026-02-22T23:27:57.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.517+0000 7fcf72ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:27:57.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.517+0000 7fcf72ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:27:57.520 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:27:57.520 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:27:57.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 10 librados: watch_flush enter 2026-02-22T23:27:57.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 10 librados: watch_flush exit 2026-02-22T23:27:57.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 20 client.5055.objecter shutdown clearing up homeless session... 2026-02-22T23:27:57.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 10 client.5055.objecter successfully canceled tick 2026-02-22T23:27:57.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf5c078e50 msgr2=0x7fcf5c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:57.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf5c078e50 0x7fcf5c07b310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c039000 tx=0x7fcf7c033000 comp rx=0 tx=0).stop 2026-02-22T23:27:57.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 10 monclient: shutdown 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c15a5e0 msgr2=0x7fcf8c14c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c15a5e0 0x7fcf8c14c250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800efc0 tx=0x7fcf8800c490 comp rx=0 tx=0).stop 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 shutdown_connections 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcf5c078e50 0x7fcf5c07b310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcf8c15a5e0 0x7fcf8c14c250 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcf8c0c99a0 0x7fcf8c1f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 --2- 10.20.193.6:0/940463012 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcf8c06ff20 0x7fcf8c1f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 >> 10.20.193.6:0/940463012 conn(0x7fcf8c0cd780 msgr2=0x7fcf8c14e580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 shutdown_connections 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 -- 10.20.193.6:0/940463012 wait complete. 2026-02-22T23:27:57.523 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:27:57.519+0000 7fcf94355640 1 librados: shutdown 2026-02-22T23:28:00.530 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:00.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.597+0000 7f87f901a640 10 monclient: get_monmap_and_config 2026-02-22T23:28:00.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.597+0000 7f87f901a640 10 monclient: build_initial_monmap 2026-02-22T23:28:00.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.597+0000 7f87f901a640 10 monclient: monmap: 2026-02-22T23:28:00.600 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:00.600 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:00.599189+0000 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:00.599189+0000 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:00.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.597+0000 7f87f901a640 1 Processor -- start 2026-02-22T23:28:00.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 1 -- start start 2026-02-22T23:28:00.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient: init 2026-02-22T23:28:00.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:00.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient: _add_conns ranks=[0,3,5,4,1,2] 2026-02-22T23:28:00.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f87f406ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:00.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f87f40c99a0 0x7f87f4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:00.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient(hunting): picked mon.noname-a con 0x7f87f40c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f87f415a5e0 0x7f87f414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient(hunting): picked mon.noname-c con 0x7f87f415a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f40c7e90 con 0x7f87f406ff20 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f87f4159a40 con 0x7f87f40c99a0 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f87f40c8170 con 0x7f87f415a5e0 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f901a640 10 monclient(hunting): authenticate will time out at 1355.326533s 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f2575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f87f40c99a0 0x7f87f4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.598+0000 7f87f3577640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f87f415a5e0 0x7f87f414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f87f40c99a0 0x7f87f4070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41352/0 (socket says 10.20.193.6:41352) 2026-02-22T23:28:00.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 1 -- 10.20.193.6:0/373071510 learned_addr learned my addr 10.20.193.6:0/373071510 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 10 monclient(hunting): get_auth_request con 0x7f87f40c99a0 auth_method 0 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f3577640 10 monclient(hunting): get_auth_request con 0x7f87f415a5e0 auth_method 0 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f3577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f3577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f3577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:00.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f3577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f3577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.599+0000 7f87f3577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 10 monclient(hunting): handle_auth_done global_id 5147 payload 274 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 1 -- 10.20.193.6:0/373071510 >> v2:10.20.193.6:3300/0 conn(0x7f87f40c99a0 msgr2=0x7f87f4070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 1 --2- 10.20.193.6:0/373071510 >> v2:10.20.193.6:3300/0 conn(0x7f87f40c99a0 0x7f87f4070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 1 -- 10.20.193.6:0/373071510 >> v1:10.20.193.6:6789/0 conn(0x7f87f406ff20 legacy=0x7f87f4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 10 monclient: _finish_hunting 0 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 1 monclient: found mon.noname-c 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.600+0000 7f87f3577640 1 -- 10.20.193.6:0/373071510 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f40c8450 con 0x7f87f415a5e0 2026-02-22T23:28:00.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.601+0000 7f87f3577640 1 --2- 10.20.193.6:0/373071510 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f415a5e0 0x7f87f414fe10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009d90 tx=0x7f87e402f1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ce2b8ac5e2221a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:00.613 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 1 -- 10.20.193.6:0/373071510 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e403c070 con 0x7f87f415a5e0 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: dump: 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:00.614 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: _finish_auth 0 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: _check_auth_tickets 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 1 -- 10.20.193.6:0/373071510 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e402fe10 con 0x7f87f415a5e0 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 5 monclient: authenticate success, global_id 5147 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 1 -- 10.20.193.6:0/373071510 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4037660 con 0x7f87f415a5e0 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:00.615 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f1d74640 10 monclient: dump: 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 4 set_mon_vals no callback set 2026-02-22T23:28:00.616 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 10 monclient: shutdown 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 -- 10.20.193.6:0/373071510 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f415a5e0 msgr2=0x7f87f414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 --2- 10.20.193.6:0/373071510 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f415a5e0 0x7f87f414fe10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009d90 tx=0x7f87e402f1e0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 -- 10.20.193.6:0/373071510 shutdown_connections 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 --2- 10.20.193.6:0/373071510 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f415a5e0 0x7f87f414fe10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 --2- 10.20.193.6:0/373071510 >> v2:10.20.193.6:3300/0 conn(0x7f87f40c99a0 0x7f87f4070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 -- 10.20.193.6:0/373071510 >> 10.20.193.6:0/373071510 conn(0x7f87f40cd780 msgr2=0x7f87f40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 -- 10.20.193.6:0/373071510 shutdown_connections 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 1 -- 10.20.193.6:0/373071510 wait complete. 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 10 monclient: build_initial_monmap 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.603+0000 7f87f901a640 10 monclient: monmap: 2026-02-22T23:28:00.617 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 librados: starting msgr at 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 librados: starting objecter 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 Processor -- start 2026-02-22T23:28:00.618 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 -- start start 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 librados: setting wanted keys 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 librados: calling monclient init 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient: init 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f406ff20 0x7f87f41f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): picked mon.noname-c con 0x7f87f406ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f87f40c99a0 0x7f87f4155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): picked mon.noname-a con 0x7f87f40c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f87f415a5e0 0x7f87f41f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): picked mon.noname-b con 0x7f87f415a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:00.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4150350 con 0x7f87f40c99a0 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f87f40c8a10 con 0x7f87f415a5e0 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f87f41504d0 con 0x7f87f406ff20 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f901a640 10 monclient(hunting): authenticate will time out at 1355.332677s 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f2575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f87f40c99a0 0x7f87f4155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.604+0000 7f87f2d76640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f406ff20 0x7f87f41f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f87f40c99a0 0x7f87f4155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41358/0 (socket says 10.20.193.6:41358) 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 1 -- 10.20.193.6:0/1583363213 learned_addr learned my addr 10.20.193.6:0/1583363213 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:00.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 10 monclient(hunting): get_auth_request con 0x7f87f40c99a0 auth_method 0 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2d76640 10 monclient(hunting): get_auth_request con 0x7f87f406ff20 auth_method 0 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f87f415a5e0 0x7f87f41f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:00.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f2d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 10 monclient(hunting): get_auth_request con 0x7f87f415a5e0 auth_method 0 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.605+0000 7f87f3577640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 10 monclient(hunting): handle_auth_done global_id 5061 payload 824 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 1 -- 10.20.193.6:0/1583363213 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f406ff20 msgr2=0x7f87f41f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f406ff20 0x7f87f41f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 1 -- 10.20.193.6:0/1583363213 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f87f415a5e0 msgr2=0x7f87f41f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f87f415a5e0 0x7f87f41f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 10 monclient: _finish_hunting 0 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 1 monclient: found mon.noname-a 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.606+0000 7f87f2575640 1 -- 10.20.193.6:0/1583363213 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f41543f0 con 0x7f87f40c99a0 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.607+0000 7f87f2575640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f87f40c99a0 0x7f87f4155770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00e960 tx=0x7f87dc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.607+0000 7f87d37fe640 10 client.?.objecter ms_handle_connect 0x7f87f40c99a0 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.607+0000 7f87d37fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 1 -- 10.20.193.6:0/1583363213 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc017070 con 0x7f87f40c99a0 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:00.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: dump: 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:00.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 1 -- 10.20.193.6:0/1583363213 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87dc00ca00 con 0x7f87f40c99a0 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 1 -- 10.20.193.6:0/1583363213 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00cdf0 con 0x7f87f40c99a0 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87d37fe640 10 monclient: dump: 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:00.625 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f1573640 4 set_mon_vals no callback set 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 5 monclient: authenticate success, global_id 5061 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 monclient: _renew_subs 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f41546d0 con 0x7f87f40c99a0 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 client.5061.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 monclient: _renew_subs 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f4154c90 con 0x7f87f40c99a0 2026-02-22T23:28:00.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 1 librados: init done 2026-02-22T23:28:00.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:00.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:00.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:00.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.609+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f4150860 con 0x7f87f40c99a0 2026-02-22T23:28:00.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.620+0000 7f87d37fe640 1 -- 10.20.193.6:0/1583363213 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f87dc013470 con 0x7f87f40c99a0 2026-02-22T23:28:00.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.621+0000 7f87d37fe640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f87c0078e00 0x7f87c007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:00.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.621+0000 7f87f2d76640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f87c0078e00 0x7f87c007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:00.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.621+0000 7f87f2d76640 10 monclient: get_auth_request con 0x7f87c0078e00 auth_method 0 2026-02-22T23:28:00.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.622+0000 7f87f2d76640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f87c0078e00 0x7f87c007b2c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f87e80046e0 tx=0x7f87e80091c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:00.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.622+0000 7f87d37fe640 10 client.5061.objecter ms_handle_connect 0x7f87c0078e00 2026-02-22T23:28:00.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.622+0000 7f87d37fe640 1 -- 10.20.193.6:0/1583363213 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f87dc05e470 con 0x7f87f40c99a0 2026-02-22T23:28:00.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.622+0000 7f87d37fe640 10 client.5061.objecter ms_dispatch2 0x7f87f4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:00.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.622+0000 7f87d37fe640 3 client.5061.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:00.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.622+0000 7f87d37fe640 3 client.5061.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:00.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.622+0000 7f87d37fe640 20 client.5061.objecter dump_active .. 0 homeless 2026-02-22T23:28:00.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.720+0000 7f87d37fe640 1 -- 10.20.193.6:0/1583363213 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f87dc062430 con 0x7f87f40c99a0 2026-02-22T23:28:00.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.720+0000 7f87d37fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:00.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.720+0000 7f87d37fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:00.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.820+0000 7f87f901a640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:00.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.820+0000 7f87f901a640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:00.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.820+0000 7f87f901a640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:00.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.820+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f87f40c8c50 con 0x7f87f40c99a0 2026-02-22T23:28:00.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.822+0000 7f87d37fe640 1 -- 10.20.193.6:0/1583363213 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f87dc066730 con 0x7f87f40c99a0 2026-02-22T23:28:00.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.822+0000 7f87d37fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:00.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.822+0000 7f87d37fe640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:00.825 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:00.825 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 10 librados: watch_flush enter 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 10 librados: watch_flush exit 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 20 client.5061.objecter shutdown clearing up homeless session... 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 10 client.5061.objecter successfully canceled tick 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f87c0078e00 msgr2=0x7f87c007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f87c0078e00 0x7f87c007b2c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f87e80046e0 tx=0x7f87e80091c0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 10 monclient: shutdown 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:00.827 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f87f40c99a0 msgr2=0x7f87f4155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f87f40c99a0 0x7f87f4155770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00e960 tx=0x7f87dc00ee30 comp rx=0 tx=0).stop 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 shutdown_connections 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f87c0078e00 0x7f87c007b2c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f87f415a5e0 0x7f87f41f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f87f40c99a0 0x7f87f4155770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 --2- 10.20.193.6:0/1583363213 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f87f406ff20 0x7f87f41f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 >> 10.20.193.6:0/1583363213 conn(0x7f87f40cd780 msgr2=0x7f87f4072480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 shutdown_connections 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 -- 10.20.193.6:0/1583363213 wait complete. 2026-02-22T23:28:00.828 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:00.823+0000 7f87f901a640 1 librados: shutdown 2026-02-22T23:28:03.835 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:03.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.899+0000 7f196e7eb640 10 monclient: get_monmap_and_config 2026-02-22T23:28:03.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.899+0000 7f196e7eb640 10 monclient: build_initial_monmap 2026-02-22T23:28:03.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.899+0000 7f196e7eb640 10 monclient: monmap: 2026-02-22T23:28:03.902 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:03.901526+0000 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:03.901526+0000 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:03.903 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:03.904 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:03.904 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:03.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 1 Processor -- start 2026-02-22T23:28:03.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 1 -- start start 2026-02-22T23:28:03.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient: init 2026-02-22T23:28:03.906 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient: _add_conns ranks=[5,3,2,4,1,0] 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f196806ff20 0x7f1968070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient(hunting): picked mon.noname-c con 0x7f196806ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f19680c99a0 0x7f1968159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient(hunting): picked mon.noname-a con 0x7f19680c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f196815a410 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196815a860 con 0x7f196815a410 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1968159a40 con 0x7f19680c99a0 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f19680c7e90 con 0x7f196806ff20 2026-02-22T23:28:03.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.900+0000 7f196e7eb640 10 monclient(hunting): authenticate will time out at 1358.628984s 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f19680c99a0 0x7f1968159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f19680c99a0 0x7f1968159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41360/0 (socket says 10.20.193.6:41360) 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 1 -- 10.20.193.6:0/2168441605 learned_addr learned my addr 10.20.193.6:0/2168441605 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 10 monclient(hunting): get_auth_request con 0x7f19680c99a0 auth_method 0 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.901+0000 7f19677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:03.908 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 10 monclient(hunting): handle_auth_done global_id 5064 payload 274 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 1 -- 10.20.193.6:0/2168441605 >> v2:10.20.193.80:3300/0 conn(0x7f196806ff20 msgr2=0x7f1968070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 1 --2- 10.20.193.6:0/2168441605 >> v2:10.20.193.80:3300/0 conn(0x7f196806ff20 0x7f1968070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 1 -- 10.20.193.6:0/2168441605 >> v1:10.20.193.80:6789/0 conn(0x7f196815a410 legacy=0x7f196814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 10 monclient: _finish_hunting 0 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 1 monclient: found mon.noname-a 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.902+0000 7f19677fe640 1 -- 10.20.193.6:0/2168441605 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19680c8170 con 0x7f19680c99a0 2026-02-22T23:28:03.909 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.903+0000 7f19677fe640 1 --2- 10.20.193.6:0/2168441605 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f1968159ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1958009970 tx=0x7f195802eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadca8ef0e3e82a9 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 1 -- 10.20.193.6:0/2168441605 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195803c070 con 0x7f19680c99a0 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: dump: 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.915 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 1 -- 10.20.193.6:0/2168441605 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195802fb20 con 0x7f19680c99a0 2026-02-22T23:28:03.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 1 -- 10.20.193.6:0/2168441605 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958037440 con 0x7f19680c99a0 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f1966ffd640 10 monclient: dump: 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:03.917 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 5 monclient: authenticate success, global_id 5064 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 4 set_mon_vals no callback set 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 10 monclient: shutdown 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 -- 10.20.193.6:0/2168441605 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 msgr2=0x7f1968159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 --2- 10.20.193.6:0/2168441605 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f1968159ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1958009970 tx=0x7f195802eef0 comp rx=0 tx=0).stop 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 -- 10.20.193.6:0/2168441605 shutdown_connections 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 --2- 10.20.193.6:0/2168441605 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f1968159ed0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:03.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 --2- 10.20.193.6:0/2168441605 >> v2:10.20.193.80:3300/0 conn(0x7f196806ff20 0x7f1968070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 -- 10.20.193.6:0/2168441605 >> 10.20.193.6:0/2168441605 conn(0x7f19680cd780 msgr2=0x7f19680cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 -- 10.20.193.6:0/2168441605 shutdown_connections 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.905+0000 7f196e7eb640 1 -- 10.20.193.6:0/2168441605 wait complete. 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient: build_initial_monmap 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient: monmap: 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:03.919 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 librados: starting msgr at 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 librados: starting objecter 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 Processor -- start 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 -- start start 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 librados: setting wanted keys 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 librados: calling monclient init 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient: init 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:03.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f196806ff20 0x7f19681f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): picked mon.noname-c con 0x7f196806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f19681f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): picked mon.noname-a con 0x7f19680c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f196815a410 0x7f19681f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): picked mon.noname-b con 0x7f196815a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f19680c8a10 con 0x7f19680c99a0 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f19680c8730 con 0x7f196815a410 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1968150520 con 0x7f196806ff20 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:03.921 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.906+0000 7f196e7eb640 10 monclient(hunting): authenticate will time out at 1358.634808s 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f19681f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f19681f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:41372/0 (socket says 10.20.193.6:41372) 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 1 -- 10.20.193.6:0/109707971 learned_addr learned my addr 10.20.193.6:0/109707971 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 10 monclient(hunting): get_auth_request con 0x7f19680c99a0 auth_method 0 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f196815a410 0x7f19681f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:03.922 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f19677fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 10 monclient(hunting): get_auth_request con 0x7f196815a410 auth_method 0 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.907+0000 7f196cd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 10 monclient(hunting): handle_auth_done global_id 5067 payload 824 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 1 -- 10.20.193.6:0/109707971 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f196806ff20 msgr2=0x7f19681f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f196806ff20 0x7f19681f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 1 -- 10.20.193.6:0/109707971 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f196815a410 msgr2=0x7f19681f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f196815a410 0x7f19681f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:03.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 10 monclient: _finish_hunting 0 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 1 monclient: found mon.noname-a 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.908+0000 7f19677fe640 1 -- 10.20.193.6:0/109707971 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1968157650 con 0x7f19680c99a0 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.909+0000 7f19677fe640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f19681f6e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1958009970 tx=0x7f19580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.909+0000 7f1964ff9640 10 client.?.objecter ms_handle_connect 0x7f19680c99a0 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.909+0000 7f1964ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 1 -- 10.20.193.6:0/109707971 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958044070 con 0x7f19680c99a0 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: dump: 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:03.924 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: _finish_auth 0 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:03.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 1 -- 10.20.193.6:0/109707971 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1958040610 con 0x7f19680c99a0 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 5 monclient: authenticate success, global_id 5067 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 10 monclient: _renew_subs 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19681f5250 con 0x7f19680c99a0 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 1 -- 10.20.193.6:0/109707971 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195803c040 con 0x7f19680c99a0 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f1964ff9640 10 monclient: dump: 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.926 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f19667fc640 4 set_mon_vals no callback set 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 10 client.5067.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 10 monclient: _renew_subs 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19680cb7d0 con 0x7f19680c99a0 2026-02-22T23:28:03.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.911+0000 7f196e7eb640 1 librados: init done 2026-02-22T23:28:03.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.912+0000 7f196e7eb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:03.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.912+0000 7f196e7eb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:03.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.912+0000 7f196e7eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:03.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.912+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1968070300 con 0x7f19680c99a0 2026-02-22T23:28:03.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.950+0000 7f1964ff9640 1 -- 10.20.193.6:0/109707971 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f195803f050 con 0x7f19680c99a0 2026-02-22T23:28:03.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.950+0000 7f1964ff9640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1938078e50 0x7f193807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:03.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.950+0000 7f1967fff640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1938078e50 0x7f193807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:03.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.950+0000 7f1967fff640 10 monclient: get_auth_request con 0x7f1938078e50 auth_method 0 2026-02-22T23:28:03.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.951+0000 7f1967fff640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1938078e50 0x7f193807b310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19540098a0 tx=0x7f1954006d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:03.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:03.951+0000 7f1964ff9640 10 client.5067.objecter ms_handle_connect 0x7f1938078e50 2026-02-22T23:28:04.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.033+0000 7f1964ff9640 1 -- 10.20.193.6:0/109707971 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f1958082e30 con 0x7f19680c99a0 2026-02-22T23:28:04.036 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.033+0000 7f1964ff9640 10 client.5067.objecter ms_dispatch2 0x7f1968000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:04.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.033+0000 7f1964ff9640 3 client.5067.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:04.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.033+0000 7f1964ff9640 3 client.5067.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:04.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.034+0000 7f1964ff9640 20 client.5067.objecter dump_active .. 0 homeless 2026-02-22T23:28:04.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.034+0000 7f1964ff9640 1 -- 10.20.193.6:0/109707971 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1958086df0 con 0x7f19680c99a0 2026-02-22T23:28:04.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.034+0000 7f1964ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:04.037 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.034+0000 7f1964ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:04.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.135+0000 7f196e7eb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:04.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.135+0000 7f196e7eb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:04.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.135+0000 7f196e7eb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:04.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.135+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f19680728e0 con 0x7f19680c99a0 2026-02-22T23:28:04.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.138+0000 7f1964ff9640 1 -- 10.20.193.6:0/109707971 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f195808b0f0 con 0x7f19680c99a0 2026-02-22T23:28:04.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.138+0000 7f1964ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:04.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.138+0000 7f1964ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:04.141 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:04.141 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:04.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 10 librados: watch_flush enter 2026-02-22T23:28:04.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 10 librados: watch_flush exit 2026-02-22T23:28:04.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 20 client.5067.objecter shutdown clearing up homeless session... 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 10 client.5067.objecter successfully canceled tick 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1938078e50 msgr2=0x7f193807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1938078e50 0x7f193807b310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19540098a0 tx=0x7f1954006d90 comp rx=0 tx=0).stop 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 10 monclient: shutdown 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 msgr2=0x7f19681f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f19681f6e10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1958009970 tx=0x7f19580047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 shutdown_connections 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1938078e50 0x7f193807b310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f196815a410 0x7f19681f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f19680c99a0 0x7f19681f6e10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:04.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 --2- 10.20.193.6:0/109707971 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f196806ff20 0x7f19681f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:04.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 >> 10.20.193.6:0/109707971 conn(0x7f19680cd780 msgr2=0x7f1968072bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:04.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 shutdown_connections 2026-02-22T23:28:04.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 -- 10.20.193.6:0/109707971 wait complete. 2026-02-22T23:28:04.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:04.139+0000 7f196e7eb640 1 librados: shutdown 2026-02-22T23:28:07.151 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:07.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.217+0000 7f74a3990640 10 monclient: get_monmap_and_config 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.217+0000 7f74a3990640 10 monclient: build_initial_monmap 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.217+0000 7f74a3990640 10 monclient: monmap: 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:07.219593+0000 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:07.219593+0000 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:07.221 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 1 Processor -- start 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 1 -- start start 2026-02-22T23:28:07.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient: init 2026-02-22T23:28:07.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:07.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient: _add_conns ranks=[4,1,5,3,0,2] 2026-02-22T23:28:07.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f749c06ff20 0x7f749c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:07.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient(hunting): picked mon.noname-b con 0x7f749c06ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:07.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f749c0c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f749c15a5e0 0x7f749c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient(hunting): picked mon.noname-c con 0x7f749c15a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f749c0c7e90 con 0x7f749c0c99a0 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f749c159a40 con 0x7f749c06ff20 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f749c0c8170 con 0x7f749c15a5e0 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.218+0000 7f74a3990640 10 monclient(hunting): authenticate will time out at 1361.946961s 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1f06640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f749c15a5e0 0x7f749c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:07.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1705640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f749c06ff20 0x7f749c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1f06640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f749c15a5e0 0x7f749c14fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:35162/0 (socket says 10.20.193.6:35162) 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1f06640 1 -- 10.20.193.6:0/461976912 learned_addr learned my addr 10.20.193.6:0/461976912 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1f06640 10 monclient(hunting): get_auth_request con 0x7f749c15a5e0 auth_method 0 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1f06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1f06640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1f06640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1705640 10 monclient(hunting): get_auth_request con 0x7f749c06ff20 auth_method 0 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1705640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1705640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.219+0000 7f74a1705640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:07.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.220+0000 7f74a1f06640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.220+0000 7f74a1f06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.220+0000 7f74a1f06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.220+0000 7f74a1705640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.220+0000 7f74a1705640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.220+0000 7f74a1705640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 10 monclient(hunting): handle_auth_done global_id 5153 payload 274 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 1 -- 10.20.193.6:0/461976912 >> v2:10.20.193.40:3300/0 conn(0x7f749c06ff20 msgr2=0x7f749c070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 1 --2- 10.20.193.6:0/461976912 >> v2:10.20.193.40:3300/0 conn(0x7f749c06ff20 0x7f749c070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 1 -- 10.20.193.6:0/461976912 >> v1:10.20.193.40:6789/0 conn(0x7f749c0c99a0 legacy=0x7f749c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:07.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 10 monclient: _finish_hunting 0 2026-02-22T23:28:07.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 1 monclient: found mon.noname-c 2026-02-22T23:28:07.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:07.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:07.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.221+0000 7f74a1f06640 1 -- 10.20.193.6:0/461976912 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c0c8450 con 0x7f749c15a5e0 2026-02-22T23:28:07.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.222+0000 7f74a1f06640 1 --2- 10.20.193.6:0/461976912 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c15a5e0 0x7f749c14fe10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7498009e50 tx=0x7f749802f2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46bdae408300af4e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:07.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 1 -- 10.20.193.6:0/461976912 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749802fcc0 con 0x7f749c15a5e0 2026-02-22T23:28:07.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:07.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:07.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: dump: 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: _finish_auth 0 2026-02-22T23:28:07.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 1 -- 10.20.193.6:0/461976912 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74980373f0 con 0x7f749c15a5e0 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f74a3990640 5 monclient: authenticate success, global_id 5153 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 1 -- 10.20.193.6:0/461976912 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749802fe60 con 0x7f749c15a5e0 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f748bfff640 10 monclient: dump: 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.238 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f74a3990640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.223+0000 7f74a3990640 4 set_mon_vals no callback set 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 10 monclient: shutdown 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 -- 10.20.193.6:0/461976912 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c15a5e0 msgr2=0x7f749c14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 --2- 10.20.193.6:0/461976912 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c15a5e0 0x7f749c14fe10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7498009e50 tx=0x7f749802f2a0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 -- 10.20.193.6:0/461976912 shutdown_connections 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 --2- 10.20.193.6:0/461976912 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c15a5e0 0x7f749c14fe10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 --2- 10.20.193.6:0/461976912 >> v2:10.20.193.40:3300/0 conn(0x7f749c06ff20 0x7f749c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 -- 10.20.193.6:0/461976912 >> 10.20.193.6:0/461976912 conn(0x7f749c0cd780 msgr2=0x7f749c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 -- 10.20.193.6:0/461976912 shutdown_connections 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 1 -- 10.20.193.6:0/461976912 wait complete. 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 10 monclient: build_initial_monmap 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.224+0000 7f74a3990640 10 monclient: monmap: 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:07.240 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 librados: starting msgr at 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 librados: starting objecter 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 Processor -- start 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 -- start start 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 librados: setting wanted keys 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 librados: calling monclient init 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient: init 2026-02-22T23:28:07.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f749c06ff20 0x7f749c1f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): picked mon.noname-b con 0x7f749c06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c0c99a0 0x7f749c1f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): picked mon.noname-c con 0x7f749c0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f749c15a5e0 0x7f749c0c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): picked mon.noname-a con 0x7f749c15a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f749c150350 con 0x7f749c15a5e0 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f749c0c8a10 con 0x7f749c06ff20 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:07.242 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1504d0 con 0x7f749c0c99a0 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a3990640 10 monclient(hunting): authenticate will time out at 1361.953723s 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.225+0000 7f74a0f04640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c0c99a0 0x7f749c1f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c0c99a0 0x7f749c1f4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:35172/0 (socket says 10.20.193.6:35172) 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 1 -- 10.20.193.6:0/4138664563 learned_addr learned my addr 10.20.193.6:0/4138664563 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 10 monclient(hunting): get_auth_request con 0x7f749c0c99a0 auth_method 0 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f749c06ff20 0x7f749c1f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f749c15a5e0 0x7f749c0c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:07.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 10 monclient(hunting): get_auth_request con 0x7f749c06ff20 auth_method 0 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a0f04640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 10 monclient(hunting): get_auth_request con 0x7f749c15a5e0 auth_method 0 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:07.244 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1705640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.226+0000 7f74a1f06640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 10 monclient(hunting): handle_auth_done global_id 5156 payload 824 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 1 -- 10.20.193.6:0/4138664563 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f749c06ff20 msgr2=0x7f749c1f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f749c06ff20 0x7f749c1f0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 1 -- 10.20.193.6:0/4138664563 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f749c15a5e0 msgr2=0x7f749c0c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f749c15a5e0 0x7f749c0c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 10 monclient: _finish_hunting 0 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 1 monclient: found mon.noname-c 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:07.245 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.227+0000 7f74a0f04640 1 -- 10.20.193.6:0/4138664563 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c1f4e10 con 0x7f749c0c99a0 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.228+0000 7f74a0f04640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c0c99a0 0x7f749c1f4440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f749000b570 tx=0x7f749000ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.228+0000 7f7489ffb640 10 client.?.objecter ms_handle_connect 0x7f749c0c99a0 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.228+0000 7f7489ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f7489ffb640 1 -- 10.20.193.6:0/4138664563 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490013020 con 0x7f749c0c99a0 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f7489ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f7489ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f7489ffb640 10 monclient: dump: 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:07.246 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f7489ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f7489ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f7489ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 5 monclient: authenticate success, global_id 5156 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 monclient: _renew_subs 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c1f50f0 con 0x7f749c0c99a0 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 client.5156.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:07.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 monclient: _renew_subs 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c1f56b0 con 0x7f749c0c99a0 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 1 librados: init done 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.229+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c150860 con 0x7f749c0c99a0 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.230+0000 7f7489ffb640 1 -- 10.20.193.6:0/4138664563 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7490004480 con 0x7f749c0c99a0 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.230+0000 7f7489ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.230+0000 7f7489ffb640 1 -- 10.20.193.6:0/4138664563 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490009380 con 0x7f749c0c99a0 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.230+0000 7f7489ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:07.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.230+0000 7f7489ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.230+0000 7f7489ffb640 10 monclient: dump: 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:07.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.230+0000 7f748b7fe640 4 set_mon_vals no callback set 2026-02-22T23:28:07.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.335+0000 7f7489ffb640 1 -- 10.20.193.6:0/4138664563 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f749000f460 con 0x7f749c0c99a0 2026-02-22T23:28:07.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.335+0000 7f7489ffb640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f746c078e00 0x7f746c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:07.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.336+0000 7f74a1705640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f746c078e00 0x7f746c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:07.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.336+0000 7f74a1705640 10 monclient: get_auth_request con 0x7f746c078e00 auth_method 0 2026-02-22T23:28:07.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.336+0000 7f74a1705640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f746c078e00 0x7f746c07b2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f748c009c40 tx=0x7f748c009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:07.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.336+0000 7f7489ffb640 10 client.5156.objecter ms_handle_connect 0x7f746c078e00 2026-02-22T23:28:07.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.337+0000 7f7489ffb640 1 -- 10.20.193.6:0/4138664563 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f7490004630 con 0x7f749c0c99a0 2026-02-22T23:28:07.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.337+0000 7f7489ffb640 10 client.5156.objecter ms_dispatch2 0x7f749c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:07.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.337+0000 7f7489ffb640 3 client.5156.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:07.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.337+0000 7f7489ffb640 3 client.5156.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:07.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.337+0000 7f7489ffb640 20 client.5156.objecter dump_active .. 0 homeless 2026-02-22T23:28:07.345 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.343+0000 7f7489ffb640 1 -- 10.20.193.6:0/4138664563 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f749001d020 con 0x7f749c0c99a0 2026-02-22T23:28:07.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.343+0000 7f7489ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:07.346 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.343+0000 7f7489ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:07.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.444+0000 7f74a3990640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:07.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.444+0000 7f74a3990640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:07.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.444+0000 7f74a3990640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:07.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.444+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f749c0c59c0 con 0x7f749c0c99a0 2026-02-22T23:28:07.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f7489ffb640 1 -- 10.20.193.6:0/4138664563 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f7490061f20 con 0x7f749c0c99a0 2026-02-22T23:28:07.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f7489ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:07.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f7489ffb640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:07.449 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:07.449 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 10 librados: watch_flush enter 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 10 librados: watch_flush exit 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 20 client.5156.objecter shutdown clearing up homeless session... 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 10 client.5156.objecter successfully canceled tick 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f746c078e00 msgr2=0x7f746c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f746c078e00 0x7f746c07b2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f748c009c40 tx=0x7f748c009290 comp rx=0 tx=0).stop 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 10 monclient: shutdown 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c0c99a0 msgr2=0x7f749c1f4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:07.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c0c99a0 0x7f749c1f4440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f749000b570 tx=0x7f749000ba40 comp rx=0 tx=0).stop 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 shutdown_connections 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f746c078e00 0x7f746c07b2c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f749c15a5e0 0x7f749c0c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f749c0c99a0 0x7f749c1f4440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 --2- 10.20.193.6:0/4138664563 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f749c06ff20 0x7f749c1f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 >> 10.20.193.6:0/4138664563 conn(0x7f749c0cd780 msgr2=0x7f749c159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 shutdown_connections 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 -- 10.20.193.6:0/4138664563 wait complete. 2026-02-22T23:28:07.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:07.447+0000 7f74a3990640 1 librados: shutdown 2026-02-22T23:28:10.458 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:10.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.525+0000 7fc212c2b640 10 monclient: get_monmap_and_config 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.525+0000 7fc212c2b640 10 monclient: build_initial_monmap 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.525+0000 7fc212c2b640 10 monclient: monmap: 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:10.526748+0000 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:10.526748+0000 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:10.528 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.525+0000 7fc212c2b640 1 Processor -- start 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.525+0000 7fc212c2b640 1 -- start start 2026-02-22T23:28:10.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.525+0000 7fc212c2b640 10 monclient: init 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient: _add_conns ranks=[3,5,1,4,2,0] 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc20c06ff20 0x7fc20c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient(hunting): picked mon.noname-a con 0x7fc20c06ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc20c0c99a0 0x7fc20c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient(hunting): picked mon.noname-c con 0x7fc20c0c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc20c15a410 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc20c15a860 con 0x7fc20c15a410 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fc20c159a40 con 0x7fc20c06ff20 2026-02-22T23:28:10.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc20c0c7e90 con 0x7fc20c0c99a0 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc212c2b640 10 monclient(hunting): authenticate will time out at 1365.254102s 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc20c06ff20 0x7fc20c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fc20c06ff20 0x7fc20c070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:42420/0 (socket says 10.20.193.6:42420) 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 1 -- 10.20.193.6:0/2669039779 learned_addr learned my addr 10.20.193.6:0/2669039779 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 10 monclient(hunting): get_auth_request con 0x7fc20c06ff20 auth_method 0 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc203fff640 1 --2- 10.20.193.6:0/2669039779 >> v2:10.20.193.80:3300/0 conn(0x7fc20c0c99a0 0x7fc20c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:10.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc203fff640 10 monclient(hunting): get_auth_request con 0x7fc20c0c99a0 auth_method 0 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc203fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc203fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc203fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.526+0000 7fc2109a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc203fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc203fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc203fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 10 monclient(hunting): handle_auth_done global_id 5073 payload 274 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 1 -- 10.20.193.6:0/2669039779 >> v2:10.20.193.80:3300/0 conn(0x7fc20c0c99a0 msgr2=0x7fc20c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 1 --2- 10.20.193.6:0/2669039779 >> v2:10.20.193.80:3300/0 conn(0x7fc20c0c99a0 0x7fc20c159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 1 -- 10.20.193.6:0/2669039779 >> v1:10.20.193.40:6789/0 conn(0x7fc20c15a410 legacy=0x7fc20c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:10.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 10 monclient: _finish_hunting 0 2026-02-22T23:28:10.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 1 monclient: found mon.noname-a 2026-02-22T23:28:10.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:10.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:10.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.527+0000 7fc2109a0640 1 -- 10.20.193.6:0/2669039779 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c0c8170 con 0x7fc20c06ff20 2026-02-22T23:28:10.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.528+0000 7fc2109a0640 1 --2- 10.20.193.6:0/2669039779 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c06ff20 0x7fc20c070300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009970 tx=0x7fc1fc02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ff4f8a6f81e528 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:10.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 1 -- 10.20.193.6:0/2669039779 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc03c070 con 0x7fc20c06ff20 2026-02-22T23:28:10.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:10.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:10.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: dump: 2026-02-22T23:28:10.541 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:10.541 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:10.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 1 -- 10.20.193.6:0/2669039779 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1fc02fbc0 con 0x7fc20c06ff20 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc212c2b640 5 monclient: authenticate success, global_id 5073 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc212c2b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 1 -- 10.20.193.6:0/2669039779 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc037440 con 0x7fc20c06ff20 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc2037fe640 10 monclient: dump: 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.543 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc212c2b640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.530+0000 7fc212c2b640 4 set_mon_vals no callback set 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 10 monclient: shutdown 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 -- 10.20.193.6:0/2669039779 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c06ff20 msgr2=0x7fc20c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 --2- 10.20.193.6:0/2669039779 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c06ff20 0x7fc20c070300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009970 tx=0x7fc1fc02eef0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 -- 10.20.193.6:0/2669039779 shutdown_connections 2026-02-22T23:28:10.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 --2- 10.20.193.6:0/2669039779 >> v2:10.20.193.80:3300/0 conn(0x7fc20c0c99a0 0x7fc20c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 --2- 10.20.193.6:0/2669039779 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c06ff20 0x7fc20c070300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 -- 10.20.193.6:0/2669039779 >> 10.20.193.6:0/2669039779 conn(0x7fc20c0cd780 msgr2=0x7fc20c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 -- 10.20.193.6:0/2669039779 shutdown_connections 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 -- 10.20.193.6:0/2669039779 wait complete. 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 10 monclient: build_initial_monmap 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 10 monclient: monmap: 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:10.545 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 librados: starting msgr at 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 librados: starting objecter 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 Processor -- start 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 -- start start 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 librados: setting wanted keys 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 1 librados: calling monclient init 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.531+0000 7fc212c2b640 10 monclient: init 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:10.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20c06ff20 0x7fc20c0c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): picked mon.noname-c con 0x7fc20c06ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c0c99a0 0x7fc20c1f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): picked mon.noname-a con 0x7fc20c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc20c15a410 0x7fc20c0c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): picked mon.noname-b con 0x7fc20c15a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0c8a10 con 0x7fc20c0c99a0 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c0c8730 con 0x7fc20c15a410 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c150520 con 0x7fc20c06ff20 2026-02-22T23:28:10.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc212c2b640 10 monclient(hunting): authenticate will time out at 1365.260295s 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c0c99a0 0x7fc20c1f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20c06ff20 0x7fc20c0c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2111a1640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc20c15a410 0x7fc20c0c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c0c99a0 0x7fc20c1f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:42424/0 (socket says 10.20.193.6:42424) 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 10 monclient(hunting): get_auth_request con 0x7fc20c06ff20 auth_method 0 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 1 -- 10.20.193.6:0/253000852 learned_addr learned my addr 10.20.193.6:0/253000852 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 10 monclient(hunting): get_auth_request con 0x7fc20c0c99a0 auth_method 0 2026-02-22T23:28:10.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2111a1640 10 monclient(hunting): get_auth_request con 0x7fc20c15a410 auth_method 0 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2111a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2111a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2111a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc203fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.532+0000 7fc2109a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:10.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.533+0000 7fc2111a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.533+0000 7fc2111a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.533+0000 7fc2111a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 10 monclient(hunting): handle_auth_done global_id 5076 payload 824 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 1 -- 10.20.193.6:0/253000852 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20c06ff20 msgr2=0x7fc20c0c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20c06ff20 0x7fc20c0c13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 1 -- 10.20.193.6:0/253000852 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc20c15a410 msgr2=0x7fc20c0c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc20c15a410 0x7fc20c0c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 1 monclient: found mon.noname-a 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc203fff640 1 -- 10.20.193.6:0/253000852 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c0c1c30 con 0x7fc20c0c99a0 2026-02-22T23:28:10.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.534+0000 7fc2109a0640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20c06ff20 0x7fc20c0c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.535+0000 7fc203fff640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c0c99a0 0x7fc20c1f4540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400ebe0 tx=0x7fc1f400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.535+0000 7fc2017fa640 10 client.?.objecter ms_handle_connect 0x7fc20c0c99a0 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.535+0000 7fc2017fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 1 -- 10.20.193.6:0/253000852 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4017070 con 0x7fc20c0c99a0 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: dump: 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:10.551 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: _finish_auth 0 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: _check_auth_tickets 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 1 -- 10.20.193.6:0/253000852 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f400cc40 con 0x7fc20c0c99a0 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 1 -- 10.20.193.6:0/253000852 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4013470 con 0x7fc20c0c99a0 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 5 monclient: authenticate success, global_id 5076 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc202ffd640 4 set_mon_vals no callback set 2026-02-22T23:28:10.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 monclient: _renew_subs 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c0c1f10 con 0x7fc20c0c99a0 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc2017fa640 10 monclient: dump: 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:10.553 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 client.5076.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 monclient: _renew_subs 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c0c24d0 con 0x7fc20c0c99a0 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 1 librados: init done 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:10.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.537+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20c070300 con 0x7fc20c0c99a0 2026-02-22T23:28:10.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.548+0000 7fc2017fa640 1 -- 10.20.193.6:0/253000852 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc1f4013610 con 0x7fc20c0c99a0 2026-02-22T23:28:10.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.548+0000 7fc2017fa640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1dc078e00 0x7fc1dc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:10.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.549+0000 7fc2109a0640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1dc078e00 0x7fc1dc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:10.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.549+0000 7fc2109a0640 10 monclient: get_auth_request con 0x7fc1dc078e00 auth_method 0 2026-02-22T23:28:10.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.549+0000 7fc2109a0640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1dc078e00 0x7fc1dc07b2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0c5380 tx=0x7fc1fc033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:10.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.549+0000 7fc2017fa640 10 client.5076.objecter ms_handle_connect 0x7fc1dc078e00 2026-02-22T23:28:10.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.551+0000 7fc2017fa640 1 -- 10.20.193.6:0/253000852 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc1f405e590 con 0x7fc20c0c99a0 2026-02-22T23:28:10.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.551+0000 7fc2017fa640 10 client.5076.objecter ms_dispatch2 0x7fc20c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:10.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.551+0000 7fc2017fa640 3 client.5076.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:10.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.551+0000 7fc2017fa640 3 client.5076.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:10.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.551+0000 7fc2017fa640 20 client.5076.objecter dump_active .. 0 homeless 2026-02-22T23:28:10.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.648+0000 7fc2017fa640 1 -- 10.20.193.6:0/253000852 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc1f4062550 con 0x7fc20c0c99a0 2026-02-22T23:28:10.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.648+0000 7fc2017fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:10.650 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.648+0000 7fc2017fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:10.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.748+0000 7fc212c2b640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:10.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.748+0000 7fc212c2b640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:10.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.748+0000 7fc212c2b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:10.751 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.748+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc20c0c7820 con 0x7fc20c0c99a0 2026-02-22T23:28:10.752 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.750+0000 7fc2017fa640 1 -- 10.20.193.6:0/253000852 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fc1f4066850 con 0x7fc20c0c99a0 2026-02-22T23:28:10.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.750+0000 7fc2017fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:10.753 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.750+0000 7fc2017fa640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:10.753 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:10.753 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:10.754 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 10 librados: watch_flush enter 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 10 librados: watch_flush exit 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 20 client.5076.objecter shutdown clearing up homeless session... 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 10 client.5076.objecter successfully canceled tick 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1dc078e00 msgr2=0x7fc1dc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1dc078e00 0x7fc1dc07b2c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0c5380 tx=0x7fc1fc033000 comp rx=0 tx=0).stop 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 10 monclient: shutdown 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c0c99a0 msgr2=0x7fc20c1f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c0c99a0 0x7fc20c1f4540 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400ebe0 tx=0x7fc1f400c520 comp rx=0 tx=0).stop 2026-02-22T23:28:10.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 shutdown_connections 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc1dc078e00 0x7fc1dc07b2c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc20c15a410 0x7fc20c0c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc20c0c99a0 0x7fc20c1f4540 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 --2- 10.20.193.6:0/253000852 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc20c06ff20 0x7fc20c0c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 >> 10.20.193.6:0/253000852 conn(0x7fc20c0cd780 msgr2=0x7fc20c073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 shutdown_connections 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 -- 10.20.193.6:0/253000852 wait complete. 2026-02-22T23:28:10.756 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:10.751+0000 7fc212c2b640 1 librados: shutdown 2026-02-22T23:28:13.379 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:57.629 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.381 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:57.630 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.389 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:06:57.639 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.407 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:57.653 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.407 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:57.619 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.408 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:57.620 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.409 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:57.655 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.416 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:06:57.663 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.417 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:06:57.628 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:13.763 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:13.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.829+0000 7f6eb02fc640 10 monclient: get_monmap_and_config 2026-02-22T23:28:13.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.829+0000 7f6eb02fc640 10 monclient: build_initial_monmap 2026-02-22T23:28:13.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.829+0000 7f6eb02fc640 10 monclient: monmap: 2026-02-22T23:28:13.831 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:13.831 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:13.830100+0000 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:13.830100+0000 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:13.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.829+0000 7f6eb02fc640 1 Processor -- start 2026-02-22T23:28:13.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.829+0000 7f6eb02fc640 1 -- start start 2026-02-22T23:28:13.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.829+0000 7f6eb02fc640 10 monclient: init 2026-02-22T23:28:13.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:13.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient: _add_conns ranks=[5,3,2,1,0,4] 2026-02-22T23:28:13.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6ea806ff20 0x7f6ea8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient(hunting): picked mon.noname-c con 0x7f6ea806ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f6ea80c99a0 0x7f6ea8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient(hunting): picked mon.noname-a con 0x7f6ea80c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f6ea815a410 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea815a860 con 0x7f6ea815a410 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea8159a40 con 0x7f6ea80c99a0 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea80c7e90 con 0x7f6ea806ff20 2026-02-22T23:28:13.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eb02fc640 10 monclient(hunting): authenticate will time out at 1368.557457s 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6eae071640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6ea806ff20 0x7f6ea8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.830+0000 7f6ead870640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f6ea80c99a0 0x7f6ea8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f6ea806ff20 0x7f6ea8070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:38688/0 (socket says 10.20.193.6:38688) 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 1 -- 10.20.193.6:0/3344111621 learned_addr learned my addr 10.20.193.6:0/3344111621 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 10 monclient(hunting): get_auth_request con 0x7f6ea806ff20 auth_method 0 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6ead870640 10 monclient(hunting): get_auth_request con 0x7f6ea80c99a0 auth_method 0 2026-02-22T23:28:13.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6ead870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6ead870640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6ead870640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6ead870640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6ead870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6ead870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.831+0000 7f6eae071640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 10 monclient(hunting): handle_auth_done global_id 5079 payload 274 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 1 -- 10.20.193.6:0/3344111621 >> v2:10.20.193.80:3300/0 conn(0x7f6ea806ff20 msgr2=0x7f6ea8070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:13.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 1 --2- 10.20.193.6:0/3344111621 >> v2:10.20.193.80:3300/0 conn(0x7f6ea806ff20 0x7f6ea8070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:13.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 1 -- 10.20.193.6:0/3344111621 >> v1:10.20.193.80:6789/0 conn(0x7f6ea815a410 legacy=0x7f6ea814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:13.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 10 monclient: _finish_hunting 0 2026-02-22T23:28:13.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 1 monclient: found mon.noname-a 2026-02-22T23:28:13.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:13.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:13.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.832+0000 7f6ead870640 1 -- 10.20.193.6:0/3344111621 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea80c8170 con 0x7f6ea80c99a0 2026-02-22T23:28:13.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.833+0000 7f6ead870640 1 --2- 10.20.193.6:0/3344111621 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea80c99a0 0x7f6ea8159ed0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0098a0 tx=0x7f6e9c02ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a00436ea6a0ae5 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 1 -- 10.20.193.6:0/3344111621 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c03c070 con 0x7f6ea80c99a0 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: dump: 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.846 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: _finish_auth 0 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: _check_auth_tickets 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 1 -- 10.20.193.6:0/3344111621 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c02fae0 con 0x7f6ea80c99a0 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:13.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 1 -- 10.20.193.6:0/3344111621 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c037440 con 0x7f6ea80c99a0 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6ead06f640 10 monclient: dump: 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:13.848 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 5 monclient: authenticate success, global_id 5079 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 4 set_mon_vals no callback set 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 10 monclient: shutdown 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 -- 10.20.193.6:0/3344111621 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea80c99a0 msgr2=0x7f6ea8159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/3344111621 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea80c99a0 0x7f6ea8159ed0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0098a0 tx=0x7f6e9c02ee10 comp rx=0 tx=0).stop 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 -- 10.20.193.6:0/3344111621 shutdown_connections 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/3344111621 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea80c99a0 0x7f6ea8159ed0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/3344111621 >> v2:10.20.193.80:3300/0 conn(0x7f6ea806ff20 0x7f6ea8070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:13.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 -- 10.20.193.6:0/3344111621 >> 10.20.193.6:0/3344111621 conn(0x7f6ea80cd780 msgr2=0x7f6ea80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 -- 10.20.193.6:0/3344111621 shutdown_connections 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 1 -- 10.20.193.6:0/3344111621 wait complete. 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 10 monclient: build_initial_monmap 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.835+0000 7f6eb02fc640 10 monclient: monmap: 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:13.850 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 librados: starting msgr at 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 librados: starting objecter 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 Processor -- start 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 -- start start 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 librados: setting wanted keys 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 librados: calling monclient init 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient: init 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea806ff20 0x7f6ea81f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:13.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): picked mon.noname-a con 0x7f6ea806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6ea80c99a0 0x7f6ea8156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): picked mon.noname-b con 0x7f6ea80c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6ea815a410 0x7f6ea81f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): picked mon.noname-c con 0x7f6ea815a410 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80c8a10 con 0x7f6ea806ff20 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80c8730 con 0x7f6ea80c99a0 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8150520 con 0x7f6ea815a410 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.836+0000 7f6eb02fc640 10 monclient(hunting): authenticate will time out at 1368.563901s 2026-02-22T23:28:13.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea806ff20 0x7f6ea81f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6ea815a410 0x7f6ea81f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea806ff20 0x7f6ea81f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:42442/0 (socket says 10.20.193.6:42442) 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 1 -- 10.20.193.6:0/1408670160 learned_addr learned my addr 10.20.193.6:0/1408670160 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 10 monclient(hunting): get_auth_request con 0x7f6ea806ff20 auth_method 0 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 10 monclient(hunting): get_auth_request con 0x7f6ea815a410 auth_method 0 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:13.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6ead870640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6ea80c99a0 0x7f6ea8156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae071640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6ead870640 10 monclient(hunting): get_auth_request con 0x7f6ea80c99a0 auth_method 0 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6ead870640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6ead870640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6ead870640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.837+0000 7f6eae872640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.838+0000 7f6ead870640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:13.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.838+0000 7f6ead870640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.838+0000 7f6ead870640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 10 monclient(hunting): handle_auth_done global_id 5171 payload 824 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 1 -- 10.20.193.6:0/1408670160 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6ea80c99a0 msgr2=0x7f6ea8156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6ea80c99a0 0x7f6ea8156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 1 -- 10.20.193.6:0/1408670160 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea806ff20 msgr2=0x7f6ea81f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea806ff20 0x7f6ea81f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 10 monclient: _finish_hunting 0 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 1 monclient: found mon.noname-c 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.839+0000 7f6eae872640 1 -- 10.20.193.6:0/1408670160 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea8150900 con 0x7f6ea815a410 2026-02-22T23:28:13.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.840+0000 7f6eae872640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6ea815a410 0x7f6ea81f2040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400cb10 tx=0x7f6ea400a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.840+0000 7f6e8effd640 10 client.?.objecter ms_handle_connect 0x7f6ea815a410 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.840+0000 7f6e8effd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 1 -- 10.20.193.6:0/1408670160 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400acb0 con 0x7f6ea815a410 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: dump: 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:13.856 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: _finish_auth 0 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 5 monclient: authenticate success, global_id 5171 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 10 monclient: _renew_subs 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea8150be0 con 0x7f6ea815a410 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 10 client.5171.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:13.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 10 monclient: _renew_subs 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81511a0 con 0x7f6ea815a410 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 1 librados: init done 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 1 -- 10.20.193.6:0/1408670160 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea400ae50 con 0x7f6ea815a410 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 1 -- 10.20.193.6:0/1408670160 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4003c70 con 0x7f6ea815a410 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eac86e640 4 set_mon_vals no callback set 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6e8effd640 10 monclient: dump: 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:13.858 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.841+0000 7f6eb02fc640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.842+0000 7f6eb02fc640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.842+0000 7f6eb02fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:13.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.842+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea8070300 con 0x7f6ea815a410 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.853+0000 7f6e8effd640 1 -- 10.20.193.6:0/1408670160 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6ea40050d0 con 0x7f6ea815a410 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.853+0000 7f6e8effd640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6e78078e00 0x7f6e7807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.854+0000 7f6eae071640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6e78078e00 0x7f6e7807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.854+0000 7f6eae071640 10 monclient: get_auth_request con 0x7f6e78078e00 auth_method 0 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.854+0000 7f6e8effd640 10 client.5171.objecter ms_handle_connect 0x7f6e78078e00 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.854+0000 7f6eae071640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6e78078e00 0x7f6e7807b2c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e98006b90 tx=0x7f6e98009340 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.855+0000 7f6e8effd640 1 -- 10.20.193.6:0/1408670160 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f6ea405e640 con 0x7f6ea815a410 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.855+0000 7f6e8effd640 10 client.5171.objecter ms_dispatch2 0x7f6ea8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.855+0000 7f6e8effd640 3 client.5171.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:13.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.855+0000 7f6e8effd640 3 client.5171.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:13.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.855+0000 7f6e8effd640 20 client.5171.objecter dump_active .. 0 homeless 2026-02-22T23:28:13.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.953+0000 7f6e8effd640 1 -- 10.20.193.6:0/1408670160 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6ea4062600 con 0x7f6ea815a410 2026-02-22T23:28:13.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.953+0000 7f6e8effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:13.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:13.953+0000 7f6e8effd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:14.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.052+0000 7f6eb02fc640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:14.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.052+0000 7f6eb02fc640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:14.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.052+0000 7f6eb02fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:14.054 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.052+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6ea8150190 con 0x7f6ea815a410 2026-02-22T23:28:14.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.054+0000 7f6e8effd640 1 -- 10.20.193.6:0/1408670160 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f6ea4066900 con 0x7f6ea815a410 2026-02-22T23:28:14.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.054+0000 7f6e8effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:14.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.054+0000 7f6e8effd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:14.056 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:14.056 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 10 librados: watch_flush enter 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 10 librados: watch_flush exit 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 20 client.5171.objecter shutdown clearing up homeless session... 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 10 client.5171.objecter successfully canceled tick 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6e78078e00 msgr2=0x7f6e7807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6e78078e00 0x7f6e7807b2c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e98006b90 tx=0x7f6e98009340 comp rx=0 tx=0).stop 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 10 monclient: shutdown 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6ea815a410 msgr2=0x7f6ea81f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:14.058 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6ea815a410 0x7f6ea81f2040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400cb10 tx=0x7f6ea400a060 comp rx=0 tx=0).stop 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 shutdown_connections 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6e78078e00 0x7f6e7807b2c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f6ea815a410 0x7f6ea81f2040 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f6ea80c99a0 0x7f6ea8156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 --2- 10.20.193.6:0/1408670160 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f6ea806ff20 0x7f6ea81f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 >> 10.20.193.6:0/1408670160 conn(0x7f6ea80cd780 msgr2=0x7f6ea80cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 shutdown_connections 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 -- 10.20.193.6:0/1408670160 wait complete. 2026-02-22T23:28:14.059 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:14.055+0000 7f6eb02fc640 1 librados: shutdown 2026-02-22T23:28:17.066 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:17.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.136+0000 7f67f4197640 10 monclient: get_monmap_and_config 2026-02-22T23:28:17.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.136+0000 7f67f4197640 10 monclient: build_initial_monmap 2026-02-22T23:28:17.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.136+0000 7f67f4197640 10 monclient: monmap: 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:17.136792+0000 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:17.136792+0000 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:17.138 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:17.139 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.136+0000 7f67f4197640 1 Processor -- start 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.136+0000 7f67f4197640 1 -- start start 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.136+0000 7f67f4197640 10 monclient: init 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient: _add_conns ranks=[1,4,5,3,0,2] 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f67ec1527d0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f67ec152bb0 0x7f67ec0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient(hunting): picked mon.noname-b con 0x7f67ec152bb0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f67ec0c5fc0 0x7f67ec0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:17.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient(hunting): picked mon.noname-c con 0x7f67ec0c5fc0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67ec06e300 con 0x7f67ec1527d0 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec06f1f0 con 0x7f67ec152bb0 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f67ec0c6470 con 0x7f67ec0c5fc0 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f4197640 10 monclient(hunting): authenticate will time out at 1371.864234s 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f170b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f67ec152bb0 0x7f67ec0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.137+0000 7f67f270d640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f67ec0c5fc0 0x7f67ec0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f67ec152bb0 0x7f67ec0c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:45646/0 (socket says 10.20.193.6:45646) 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 1 -- 10.20.193.6:0/3327296728 learned_addr learned my addr 10.20.193.6:0/3327296728 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:17.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 10 monclient(hunting): get_auth_request con 0x7f67ec152bb0 auth_method 0 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f270d640 10 monclient(hunting): get_auth_request con 0x7f67ec0c5fc0 auth_method 0 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f270d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f270d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f270d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f270d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f270d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f270d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:17.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.138+0000 7f67f170b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 10 monclient(hunting): handle_auth_done global_id 5174 payload 274 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 1 -- 10.20.193.6:0/3327296728 >> v2:10.20.193.40:3300/0 conn(0x7f67ec152bb0 msgr2=0x7f67ec0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 1 --2- 10.20.193.6:0/3327296728 >> v2:10.20.193.40:3300/0 conn(0x7f67ec152bb0 0x7f67ec0c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 1 -- 10.20.193.6:0/3327296728 >> v1:10.20.193.40:6789/0 conn(0x7f67ec1527d0 legacy=0x7f67ec06da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 10 monclient: _finish_hunting 0 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 1 monclient: found mon.noname-c 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.139+0000 7f67f270d640 1 -- 10.20.193.6:0/3327296728 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec06e5e0 con 0x7f67ec0c5fc0 2026-02-22T23:28:17.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.140+0000 7f67f270d640 1 --2- 10.20.193.6:0/3327296728 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec0c5fc0 0x7f67ec0c8e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f67ec158940 tx=0x7f67e802f1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f52387920aae167 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f0f0a640 1 -- 10.20.193.6:0/3327296728 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e803c070 con 0x7f67ec0c5fc0 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f0f0a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f0f0a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f0f0a640 10 monclient: dump: 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:17.152 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f0f0a640 10 monclient: _finish_auth 0 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f0f0a640 10 monclient: _check_auth_tickets 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f0f0a640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f4197640 5 monclient: authenticate success, global_id 5174 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.141+0000 7f67f4197640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f0f0a640 1 -- 10.20.193.6:0/3327296728 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e802fe30 con 0x7f67ec0c5fc0 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f0f0a640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:17.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f0f0a640 1 -- 10.20.193.6:0/3327296728 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8037640 con 0x7f67ec0c5fc0 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f0f0a640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f0f0a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f0f0a640 10 monclient: dump: 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:17.154 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 4 set_mon_vals no callback set 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 10 monclient: shutdown 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 -- 10.20.193.6:0/3327296728 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec0c5fc0 msgr2=0x7f67ec0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 --2- 10.20.193.6:0/3327296728 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec0c5fc0 0x7f67ec0c8e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f67ec158940 tx=0x7f67e802f1c0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 -- 10.20.193.6:0/3327296728 shutdown_connections 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 --2- 10.20.193.6:0/3327296728 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec0c5fc0 0x7f67ec0c8e60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 --2- 10.20.193.6:0/3327296728 >> v2:10.20.193.40:3300/0 conn(0x7f67ec152bb0 0x7f67ec0c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 -- 10.20.193.6:0/3327296728 >> 10.20.193.6:0/3327296728 conn(0x7f67ec0cd780 msgr2=0x7f67ec0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 -- 10.20.193.6:0/3327296728 shutdown_connections 2026-02-22T23:28:17.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.142+0000 7f67f4197640 1 -- 10.20.193.6:0/3327296728 wait complete. 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient: build_initial_monmap 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient: monmap: 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:17.156 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 librados: starting msgr at 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 librados: starting objecter 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 Processor -- start 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 -- start start 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 librados: setting wanted keys 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 librados: calling monclient init 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient: init 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f67ec0c5fc0 0x7f67ec157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): picked mon.noname-b con 0x7f67ec0c5fc0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec1527d0 0x7f67ec0c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:17.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): picked mon.noname-c con 0x7f67ec1527d0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f67ec152bb0 0x7f67ec148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): picked mon.noname-a con 0x7f67ec152bb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec06ee80 con 0x7f67ec152bb0 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec06eba0 con 0x7f67ec0c5fc0 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0c93a0 con 0x7f67ec1527d0 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.143+0000 7f67f4197640 10 monclient(hunting): authenticate will time out at 1371.870830s 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f67ec152bb0 0x7f67ec148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec1527d0 0x7f67ec0c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:17.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f67ec0c5fc0 0x7f67ec157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f67ec152bb0 0x7f67ec148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:42446/0 (socket says 10.20.193.6:42446) 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 1 -- 10.20.193.6:0/1905379115 learned_addr learned my addr 10.20.193.6:0/1905379115 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 10 monclient(hunting): get_auth_request con 0x7f67ec152bb0 auth_method 0 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 10 monclient(hunting): get_auth_request con 0x7f67ec1527d0 auth_method 0 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 10 monclient(hunting): get_auth_request con 0x7f67ec0c5fc0 auth_method 0 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:17.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f270d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f170b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.144+0000 7f67f1f0c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 10 monclient(hunting): handle_auth_done global_id 5177 payload 824 2026-02-22T23:28:17.160 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 1 -- 10.20.193.6:0/1905379115 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f67ec0c5fc0 msgr2=0x7f67ec157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f67ec0c5fc0 0x7f67ec157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 1 -- 10.20.193.6:0/1905379115 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f67ec152bb0 msgr2=0x7f67ec148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f67ec152bb0 0x7f67ec148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 10 monclient: _finish_hunting 0 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 1 monclient: found mon.noname-c 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.145+0000 7f67f170b640 1 -- 10.20.193.6:0/1905379115 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec14a8d0 con 0x7f67ec1527d0 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.147+0000 7f67f170b640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec1527d0 0x7f67ec0c25b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e800 tx=0x7f67e000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.147+0000 7f67da7fc640 10 client.?.objecter ms_handle_connect 0x7f67ec1527d0 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.147+0000 7f67da7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:17.161 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 1 -- 10.20.193.6:0/1905379115 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0017070 con 0x7f67ec1527d0 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: dump: 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:17.162 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 5 monclient: authenticate success, global_id 5177 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 monclient: _renew_subs 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec0c1250 con 0x7f67ec1527d0 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 client.5177.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 monclient: _renew_subs 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec0c17f0 con 0x7f67ec1527d0 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 1 librados: init done 2026-02-22T23:28:17.163 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ec0c9730 con 0x7f67ec1527d0 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 1 -- 10.20.193.6:0/1905379115 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e0005a90 con 0x7f67ec1527d0 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 1 -- 10.20.193.6:0/1905379115 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e0021060 con 0x7f67ec1527d0 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67da7fc640 10 monclient: dump: 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.164 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:17.165 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:17.165 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:17.165 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:17.165 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:17.165 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:17.165 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:17.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.148+0000 7f67dbfff640 4 set_mon_vals no callback set 2026-02-22T23:28:17.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.254+0000 7f67da7fc640 1 -- 10.20.193.6:0/1905379115 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f67e0024060 con 0x7f67ec1527d0 2026-02-22T23:28:17.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.254+0000 7f67da7fc640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67bc078e00 0x7f67bc07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.254+0000 7f67f1f0c640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67bc078e00 0x7f67bc07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.254+0000 7f67f1f0c640 10 monclient: get_auth_request con 0x7f67bc078e00 auth_method 0 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.255+0000 7f67da7fc640 10 client.5177.objecter ms_handle_connect 0x7f67bc078e00 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.255+0000 7f67f1f0c640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67bc078e00 0x7f67bc07b2c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0046e0 tx=0x7f67dc0091c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.255+0000 7f67da7fc640 1 -- 10.20.193.6:0/1905379115 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f67e00978f0 con 0x7f67ec1527d0 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.255+0000 7f67da7fc640 10 client.5177.objecter ms_dispatch2 0x7f67ec000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.255+0000 7f67da7fc640 3 client.5177.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.255+0000 7f67da7fc640 3 client.5177.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:17.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.255+0000 7f67da7fc640 20 client.5177.objecter dump_active .. 0 homeless 2026-02-22T23:28:17.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.261+0000 7f67da7fc640 1 -- 10.20.193.6:0/1905379115 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f67e0061fb0 con 0x7f67ec1527d0 2026-02-22T23:28:17.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.261+0000 7f67da7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:17.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.261+0000 7f67da7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:17.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.362+0000 7f67f4197640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:17.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.362+0000 7f67f4197640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:17.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.362+0000 7f67f4197640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:17.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.362+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f67ec14efa0 con 0x7f67ec1527d0 2026-02-22T23:28:17.366 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.365+0000 7f67da7fc640 1 -- 10.20.193.6:0/1905379115 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f67e00662b0 con 0x7f67ec1527d0 2026-02-22T23:28:17.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.365+0000 7f67da7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:17.367 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.365+0000 7f67da7fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:17.367 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:17.367 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 10 librados: watch_flush enter 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 10 librados: watch_flush exit 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 20 client.5177.objecter shutdown clearing up homeless session... 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 10 client.5177.objecter successfully canceled tick 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67bc078e00 msgr2=0x7f67bc07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67bc078e00 0x7f67bc07b2c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0046e0 tx=0x7f67dc0091c0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 10 monclient: shutdown 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec1527d0 msgr2=0x7f67ec0c25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:17.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec1527d0 0x7f67ec0c25b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e800 tx=0x7f67e000ecd0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 shutdown_connections 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f67bc078e00 0x7f67bc07b2c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f67ec152bb0 0x7f67ec148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f67ec1527d0 0x7f67ec0c25b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 --2- 10.20.193.6:0/1905379115 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f67ec0c5fc0 0x7f67ec157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 >> 10.20.193.6:0/1905379115 conn(0x7f67ec0cd780 msgr2=0x7f67ec1512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 shutdown_connections 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 -- 10.20.193.6:0/1905379115 wait complete. 2026-02-22T23:28:17.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:17.366+0000 7f67f4197640 1 librados: shutdown 2026-02-22T23:28:20.377 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:20.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.446+0000 7fa1dbdeb640 10 monclient: get_monmap_and_config 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.446+0000 7fa1dbdeb640 10 monclient: build_initial_monmap 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.446+0000 7fa1dbdeb640 10 monclient: monmap: 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:20.447424+0000 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:20.447424+0000 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.447+0000 7fa1dbdeb640 1 Processor -- start 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.447+0000 7fa1dbdeb640 1 -- start start 2026-02-22T23:28:20.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.447+0000 7fa1dbdeb640 10 monclient: init 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.447+0000 7fa1dbdeb640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.447+0000 7fa1dbdeb640 10 monclient: _add_conns ranks=[3,2,1,4,0,5] 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa1d4155c70 0x7fa1d415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 10 monclient(hunting): picked mon.noname-a con 0x7fa1d4155c70 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa1d40ccb30 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa1d4156050 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d415cc60 con 0x7fa1d4156050 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d4072e90 con 0x7fa1d40ccb30 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d406f440 con 0x7fa1d4155c70 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1dbdeb640 10 monclient(hunting): authenticate will time out at 1375.175054s 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1da361640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fa1d4156050 0x7fa1d415c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:43926/0 (socket says 10.20.193.6:43926) 2026-02-22T23:28:20.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.448+0000 7fa1da361640 1 -- 10.20.193.6:0/1498786375 learned_addr learned my addr 10.20.193.6:0/1498786375 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 1 --2- 10.20.193.6:0/1498786375 >> v2:10.20.193.6:3300/0 conn(0x7fa1d4155c70 0x7fa1d415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 10 monclient(hunting): get_auth_request con 0x7fa1d4155c70 auth_method 0 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:20.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.462+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:20.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_done global_id 5088 payload 274 2026-02-22T23:28:20.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 1 -- 10.20.193.6:0/1498786375 >> v1:10.20.193.80:6789/0 conn(0x7fa1d40ccb30 legacy=0x7fa1d415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:20.465 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 1 -- 10.20.193.6:0/1498786375 >> v1:10.20.193.40:6789/0 conn(0x7fa1d4156050 legacy=0x7fa1d415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:20.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 10 monclient: _finish_hunting 0 2026-02-22T23:28:20.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 1 monclient: found mon.noname-a 2026-02-22T23:28:20.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:20.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:20.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.463+0000 7fa1d9b60640 1 -- 10.20.193.6:0/1498786375 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d40cbcb0 con 0x7fa1d4155c70 2026-02-22T23:28:20.466 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.464+0000 7fa1d9b60640 1 --2- 10.20.193.6:0/1498786375 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d4155c70 0x7fa1d415d2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40098a0 tx=0x7fa1c402ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cbdaef07f747c6 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:20.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 1 -- 10.20.193.6:0/1498786375 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c403c070 con 0x7fa1d4155c70 2026-02-22T23:28:20.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:20.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:20.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: dump: 2026-02-22T23:28:20.470 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:20.470 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:20.470 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: _finish_auth 0 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: _check_auth_tickets 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 5 monclient: authenticate success, global_id 5088 2026-02-22T23:28:20.471 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 1 -- 10.20.193.6:0/1498786375 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c402fb70 con 0x7fa1d4155c70 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 1 -- 10.20.193.6:0/1498786375 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4037440 con 0x7fa1d4155c70 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 4 set_mon_vals no callback set 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1d8b5e640 10 monclient: dump: 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.472 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 10 monclient: shutdown 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/1498786375 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d4155c70 msgr2=0x7fa1d415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/1498786375 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d4155c70 0x7fa1d415d2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40098a0 tx=0x7fa1c402ee60 comp rx=0 tx=0).stop 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/1498786375 shutdown_connections 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/1498786375 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d4155c70 0x7fa1d415d2c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/1498786375 >> 10.20.193.6:0/1498786375 conn(0x7fa1d40cc010 msgr2=0x7fa1d40cc420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/1498786375 shutdown_connections 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.466+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/1498786375 wait complete. 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 10 monclient: build_initial_monmap 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 10 monclient: monmap: 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:20.474 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 1 librados: starting msgr at 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 1 librados: starting objecter 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 1 Processor -- start 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 1 -- start start 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 1 librados: setting wanted keys 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 1 librados: calling monclient init 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.467+0000 7fa1dbdeb640 10 monclient: init 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d40ccb30 0x7fa1d40c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:20.475 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): picked mon.noname-a con 0x7fa1d40ccb30 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1d4155c70 0x7fa1d415e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): picked mon.noname-c con 0x7fa1d4155c70 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1d4156050 0x7fa1d41e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): picked mon.noname-b con 0x7fa1d4156050 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d406fc50 con 0x7fa1d40ccb30 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d406f970 con 0x7fa1d4156050 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d406ff30 con 0x7fa1d4155c70 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1dbdeb640 10 monclient(hunting): authenticate will time out at 1375.195103s 2026-02-22T23:28:20.476 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d40ccb30 0x7fa1d40c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d40ccb30 0x7fa1d40c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54286/0 (socket says 10.20.193.6:54286) 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 1 -- 10.20.193.6:0/2587559705 learned_addr learned my addr 10.20.193.6:0/2587559705 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 10 monclient(hunting): get_auth_request con 0x7fa1d40ccb30 auth_method 0 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:20.477 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.468+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:20.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 10 monclient(hunting): handle_auth_done global_id 5091 payload 824 2026-02-22T23:28:20.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 1 -- 10.20.193.6:0/2587559705 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1d4155c70 msgr2=0x7fa1d415e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:20.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1d4155c70 0x7fa1d415e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 1 -- 10.20.193.6:0/2587559705 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1d4156050 msgr2=0x7fa1d41e99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:20.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1d4156050 0x7fa1d41e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 10 monclient: _finish_hunting 0 2026-02-22T23:28:20.480 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 1 monclient: found mon.noname-a 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.469+0000 7fa1d9b60640 1 -- 10.20.193.6:0/2587559705 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d41e7f80 con 0x7fa1d40ccb30 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.470+0000 7fa1d9b60640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d40ccb30 0x7fa1d40c7970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009130 tx=0x7fa1c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.470+0000 7fa1c27fc640 10 client.?.objecter ms_handle_connect 0x7fa1d40ccb30 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.470+0000 7fa1c27fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 1 -- 10.20.193.6:0/2587559705 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4046020 con 0x7fa1d40ccb30 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: dump: 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:20.481 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 1 -- 10.20.193.6:0/2587559705 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4041670 con 0x7fa1d40ccb30 2026-02-22T23:28:20.482 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 1 -- 10.20.193.6:0/2587559705 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c403c040 con 0x7fa1d40ccb30 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c27fc640 10 monclient: dump: 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:20.483 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 5 monclient: authenticate success, global_id 5091 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1c3fff640 4 set_mon_vals no callback set 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 monclient: _renew_subs 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d41e8260 con 0x7fa1d40ccb30 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 client.5091.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 monclient: _renew_subs 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d40c7170 con 0x7fa1d40ccb30 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 1 librados: init done 2026-02-22T23:28:20.484 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:20.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:20.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:20.485 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.472+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d40701f0 con 0x7fa1d40ccb30 2026-02-22T23:28:20.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.483+0000 7fa1c27fc640 1 -- 10.20.193.6:0/2587559705 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa1c4041a60 con 0x7fa1d40ccb30 2026-02-22T23:28:20.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.483+0000 7fa1c27fc640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1a4078e50 0x7fa1a407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:20.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1d935f640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1a4078e50 0x7fa1a407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:20.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1d935f640 10 monclient: get_auth_request con 0x7fa1a4078e50 auth_method 0 2026-02-22T23:28:20.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1c27fc640 1 -- 10.20.193.6:0/2587559705 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fa1c4041d80 con 0x7fa1d40ccb30 2026-02-22T23:28:20.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1c27fc640 10 client.5091.objecter ms_dispatch2 0x7fa1d4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:20.486 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1c27fc640 3 client.5091.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:20.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1c27fc640 3 client.5091.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:20.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1c27fc640 20 client.5091.objecter dump_active .. 0 homeless 2026-02-22T23:28:20.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1d935f640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1a4078e50 0x7fa1a407b310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8002770 tx=0x7fa1c8007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:20.487 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.484+0000 7fa1c27fc640 10 client.5091.objecter ms_handle_connect 0x7fa1a4078e50 2026-02-22T23:28:20.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.580+0000 7fa1c27fc640 1 -- 10.20.193.6:0/2587559705 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa1c4050020 con 0x7fa1d40ccb30 2026-02-22T23:28:20.581 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.580+0000 7fa1c27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:20.582 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.580+0000 7fa1c27fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:20.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.680+0000 7fa1dbdeb640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:20.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.680+0000 7fa1dbdeb640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:20.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.680+0000 7fa1dbdeb640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:20.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.680+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa1d40cb040 con 0x7fa1d40ccb30 2026-02-22T23:28:20.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.682+0000 7fa1c27fc640 1 -- 10.20.193.6:0/2587559705 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fa1c40873b0 con 0x7fa1d40ccb30 2026-02-22T23:28:20.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.682+0000 7fa1c27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:20.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.682+0000 7fa1c27fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:20.684 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:20.684 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:20.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 10 librados: watch_flush enter 2026-02-22T23:28:20.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 10 librados: watch_flush exit 2026-02-22T23:28:20.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 20 client.5091.objecter shutdown clearing up homeless session... 2026-02-22T23:28:20.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 10 client.5091.objecter successfully canceled tick 2026-02-22T23:28:20.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1a4078e50 msgr2=0x7fa1a407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:20.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1a4078e50 0x7fa1a407b310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8002770 tx=0x7fa1c8007ab0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 10 monclient: shutdown 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d40ccb30 msgr2=0x7fa1d40c7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d40ccb30 0x7fa1d40c7970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009130 tx=0x7fa1c40047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 shutdown_connections 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa1a4078e50 0x7fa1a407b310 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa1d4156050 0x7fa1d41e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa1d4155c70 0x7fa1d415e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 --2- 10.20.193.6:0/2587559705 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa1d40ccb30 0x7fa1d40c7970 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 >> 10.20.193.6:0/2587559705 conn(0x7fa1d40cc010 msgr2=0x7fa1d40cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 shutdown_connections 2026-02-22T23:28:20.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.683+0000 7fa1dbdeb640 1 -- 10.20.193.6:0/2587559705 wait complete. 2026-02-22T23:28:20.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:20.684+0000 7fa1dbdeb640 1 librados: shutdown 2026-02-22T23:28:23.694 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:23.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.762+0000 7f86f4a06640 10 monclient: get_monmap_and_config 2026-02-22T23:28:23.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.762+0000 7f86f4a06640 10 monclient: build_initial_monmap 2026-02-22T23:28:23.764 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.762+0000 7f86f4a06640 10 monclient: monmap: 2026-02-22T23:28:23.764 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:23.764 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:23.764 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:23.763150+0000 2026-02-22T23:28:23.764 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:23.763150+0000 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.762+0000 7f86f4a06640 1 Processor -- start 2026-02-22T23:28:23.765 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.762+0000 7f86f4a06640 1 -- start start 2026-02-22T23:28:23.766 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.762+0000 7f86f4a06640 10 monclient: init 2026-02-22T23:28:23.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:23.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient: _add_conns ranks=[1,3,0,5,2,4] 2026-02-22T23:28:23.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f86f006ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:23.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f86f00c99a0 0x7f86f0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:23.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient(hunting): picked mon.noname-a con 0x7f86f00c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:23.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f86f015a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:23.768 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f00c7e90 con 0x7f86f015a5e0 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f00c8170 con 0x7f86f006ff20 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f86f0159a40 con 0x7f86f00c99a0 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86f4a06640 10 monclient(hunting): authenticate will time out at 1378.490523s 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86edd74640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f86f00c99a0 0x7f86f0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.763+0000 7f86ee575640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f86f006ff20 0x7f86f0159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:43930/0 (socket says 10.20.193.6:43930) 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f86f00c99a0 0x7f86f0070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54294/0 (socket says 10.20.193.6:54294) 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86ee575640 1 -- 10.20.193.6:0/2158160921 learned_addr learned my addr 10.20.193.6:0/2158160921 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 10 monclient(hunting): get_auth_request con 0x7f86f00c99a0 auth_method 0 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:23.769 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.764+0000 7f86edd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 10 monclient(hunting): handle_auth_done global_id 5094 payload 274 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 1 -- 10.20.193.6:0/2158160921 >> v1:10.20.193.40:6789/0 conn(0x7f86f006ff20 legacy=0x7f86f0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 1 -- 10.20.193.6:0/2158160921 >> v1:10.20.193.6:6789/0 conn(0x7f86f015a5e0 legacy=0x7f86f014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 10 monclient: _finish_hunting 0 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 1 monclient: found mon.noname-a 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:23.770 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 1 -- 10.20.193.6:0/2158160921 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f00c8450 con 0x7f86f00c99a0 2026-02-22T23:28:23.771 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.765+0000 7f86edd74640 1 --2- 10.20.193.6:0/2158160921 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f00c99a0 0x7f86f0070300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86d80098d0 tx=0x7f86d802ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06f4d3f14b6e717 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:23.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 1 -- 10.20.193.6:0/2158160921 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d803c070 con 0x7f86f00c99a0 2026-02-22T23:28:23.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:23.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:23.777 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: dump: 2026-02-22T23:28:23.777 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:23.777 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:23.777 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: _finish_auth 0 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: _check_auth_tickets 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 1 -- 10.20.193.6:0/2158160921 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d802fb00 con 0x7f86f00c99a0 2026-02-22T23:28:23.778 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 1 -- 10.20.193.6:0/2158160921 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8037440 con 0x7f86f00c99a0 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86ed573640 10 monclient: dump: 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:23.779 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 5 monclient: authenticate success, global_id 5094 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 4 set_mon_vals no callback set 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 10 monclient: shutdown 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 1 -- 10.20.193.6:0/2158160921 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f00c99a0 msgr2=0x7f86f0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 1 --2- 10.20.193.6:0/2158160921 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f00c99a0 0x7f86f0070300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f86d80098d0 tx=0x7f86d802ee80 comp rx=0 tx=0).stop 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 1 -- 10.20.193.6:0/2158160921 shutdown_connections 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 1 --2- 10.20.193.6:0/2158160921 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f00c99a0 0x7f86f0070300 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:23.780 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 1 -- 10.20.193.6:0/2158160921 >> 10.20.193.6:0/2158160921 conn(0x7f86f00cd780 msgr2=0x7f86f00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 1 -- 10.20.193.6:0/2158160921 shutdown_connections 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 1 -- 10.20.193.6:0/2158160921 wait complete. 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 10 monclient: build_initial_monmap 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.768+0000 7f86f4a06640 10 monclient: monmap: 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:23.781 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 librados: starting msgr at 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 librados: starting objecter 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 Processor -- start 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 -- start start 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 librados: setting wanted keys 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 librados: calling monclient init 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient: init 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f86f006ff20 0x7f86f014ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:23.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): picked mon.noname-b con 0x7f86f006ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f86f00c99a0 0x7f86f014d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): picked mon.noname-c con 0x7f86f00c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f015a5e0 0x7f86f00bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): picked mon.noname-a con 0x7f86f015a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0150520 con 0x7f86f015a5e0 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f86f00c8a10 con 0x7f86f006ff20 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01506a0 con 0x7f86f00c99a0 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:23.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86f4a06640 10 monclient(hunting): authenticate will time out at 1378.496647s 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.769+0000 7f86eed76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f015a5e0 0x7f86f00bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f015a5e0 0x7f86f00bc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54310/0 (socket says 10.20.193.6:54310) 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 1 -- 10.20.193.6:0/3183558919 learned_addr learned my addr 10.20.193.6:0/3183558919 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 10 monclient(hunting): get_auth_request con 0x7f86f015a5e0 auth_method 0 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f86f006ff20 0x7f86f014ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f86f00c99a0 0x7f86f014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86eed76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:23.784 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 10 monclient(hunting): get_auth_request con 0x7f86f006ff20 auth_method 0 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 10 monclient(hunting): get_auth_request con 0x7f86f00c99a0 auth_method 0 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86ee575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:23.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.770+0000 7f86edd74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 10 monclient(hunting): handle_auth_done global_id 5097 payload 824 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 1 -- 10.20.193.6:0/3183558919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f86f00c99a0 msgr2=0x7f86f014d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f86f00c99a0 0x7f86f014d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 1 -- 10.20.193.6:0/3183558919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f86f006ff20 msgr2=0x7f86f014ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f86f006ff20 0x7f86f014ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 10 monclient: _finish_hunting 0 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 1 monclient: found mon.noname-a 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.771+0000 7f86eed76640 1 -- 10.20.193.6:0/3183558919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01f74a0 con 0x7f86f015a5e0 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.772+0000 7f86eed76640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f015a5e0 0x7f86f00bc4d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b990 tx=0x7f86e000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:23.786 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.772+0000 7f86d6ffd640 10 client.?.objecter ms_handle_connect 0x7f86f015a5e0 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.772+0000 7f86d6ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 1 -- 10.20.193.6:0/3183558919 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e000c790 con 0x7f86f015a5e0 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: dump: 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:23.787 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 1 -- 10.20.193.6:0/3183558919 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e0010070 con 0x7f86f015a5e0 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 5 monclient: authenticate success, global_id 5097 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 monclient: _renew_subs 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01f6140 con 0x7f86f015a5e0 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:23.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 1 -- 10.20.193.6:0/3183558919 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e000cbb0 con 0x7f86f015a5e0 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86ecd72640 4 set_mon_vals no callback set 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86d6ffd640 10 monclient: dump: 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:23.789 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 client.5097.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 monclient: _renew_subs 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01f6700 con 0x7f86f015a5e0 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 1 librados: init done 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:23.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.774+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f0070300 con 0x7f86f015a5e0 2026-02-22T23:28:23.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.877+0000 7f86d6ffd640 1 -- 10.20.193.6:0/3183558919 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f86e0004070 con 0x7f86f015a5e0 2026-02-22T23:28:23.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.877+0000 7f86d6ffd640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f86bc078e50 0x7f86bc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:23.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.878+0000 7f86ee575640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f86bc078e50 0x7f86bc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:23.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.878+0000 7f86ee575640 10 monclient: get_auth_request con 0x7f86bc078e50 auth_method 0 2026-02-22T23:28:23.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.878+0000 7f86ee575640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f86bc078e50 0x7f86bc07b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86e4002770 tx=0x7f86e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:23.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.878+0000 7f86d6ffd640 10 client.5097.objecter ms_handle_connect 0x7f86bc078e50 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 1 -- 10.20.193.6:0/3183558919 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f86e000cd50 con 0x7f86f015a5e0 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 10 client.5097.objecter ms_dispatch2 0x7f86f0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 3 client.5097.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 3 client.5097.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 20 client.5097.objecter dump_active .. 0 homeless 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 1 -- 10.20.193.6:0/3183558919 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f86f01f74a0 con 0x7f86f015a5e0 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:23.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.885+0000 7f86d6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:23.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.985+0000 7f86f4a06640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:23.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.986+0000 7f86f4a06640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:23.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.986+0000 7f86f4a06640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:23.987 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.986+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f86f0153e20 con 0x7f86f015a5e0 2026-02-22T23:28:23.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.988+0000 7f86d6ffd640 1 -- 10.20.193.6:0/3183558919 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f86e0061b50 con 0x7f86f015a5e0 2026-02-22T23:28:23.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.988+0000 7f86d6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:23.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.988+0000 7f86d6ffd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:23.990 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:23.990 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:23.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 10 librados: watch_flush enter 2026-02-22T23:28:23.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 10 librados: watch_flush exit 2026-02-22T23:28:23.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 20 client.5097.objecter shutdown clearing up homeless session... 2026-02-22T23:28:23.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 10 client.5097.objecter successfully canceled tick 2026-02-22T23:28:23.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f86bc078e50 msgr2=0x7f86bc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:23.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f86bc078e50 0x7f86bc07b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86e4002770 tx=0x7f86e4009040 comp rx=0 tx=0).stop 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 10 monclient: shutdown 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f015a5e0 msgr2=0x7f86f00bc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f015a5e0 0x7f86f00bc4d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f86e000b990 tx=0x7f86e000be60 comp rx=0 tx=0).stop 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 shutdown_connections 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f86bc078e50 0x7f86bc07b310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f86f015a5e0 0x7f86f00bc4d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f86f00c99a0 0x7f86f014d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 --2- 10.20.193.6:0/3183558919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f86f006ff20 0x7f86f014ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 >> 10.20.193.6:0/3183558919 conn(0x7f86f00cd780 msgr2=0x7f86f014e400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 shutdown_connections 2026-02-22T23:28:23.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 -- 10.20.193.6:0/3183558919 wait complete. 2026-02-22T23:28:23.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:23.989+0000 7f86f4a06640 1 librados: shutdown 2026-02-22T23:28:27.000 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.068+0000 7fde28f14640 10 monclient: get_monmap_and_config 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.068+0000 7fde28f14640 10 monclient: build_initial_monmap 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.068+0000 7fde28f14640 10 monclient: monmap: 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:27.069289+0000 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:27.069289+0000 2026-02-22T23:28:27.070 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:27.071 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:27.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 1 Processor -- start 2026-02-22T23:28:27.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 1 -- start start 2026-02-22T23:28:27.073 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient: init 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient: _add_conns ranks=[0,1,2,5,3,4] 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fde2406ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fde24155eb0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fde2415c970 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde240725b0 con 0x7fde2406ff20 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde24072890 con 0x7fde24155eb0 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde24072b70 con 0x7fde2415c970 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.069+0000 7fde28f14640 10 monclient(hunting): authenticate will time out at 1381.796682s 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.070+0000 7fde22575640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fde2406ff20 0x7fde241557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:47852/0 (socket says 10.20.193.6:47852) 2026-02-22T23:28:27.074 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.070+0000 7fde22575640 1 -- 10.20.193.6:0/2865138381 learned_addr learned my addr 10.20.193.6:0/2865138381 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 385459536 0 0) 0x7fde240725b0 con 0x7fde2406ff20 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): _negotiate 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): my global_id is 5100 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde10003620 con 0x7fde2406ff20 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819610296 0 0) 0x7fde24072890 con 0x7fde24155eb0 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): _negotiate 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 10 monclient(hunting): my global_id is 5056 2026-02-22T23:28:27.075 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde240725b0 con 0x7fde24155eb0 2026-02-22T23:28:27.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.071+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001050022 0 0) 0x7fde24072b70 con 0x7fde2415c970 2026-02-22T23:28:27.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.072+0000 7fde21573640 10 monclient(hunting): _negotiate 2026-02-22T23:28:27.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.072+0000 7fde21573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.072+0000 7fde21573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:27.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.072+0000 7fde21573640 10 monclient(hunting): my global_id is 5183 2026-02-22T23:28:27.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.072+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde24072890 con 0x7fde2415c970 2026-02-22T23:28:27.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1955228837 0 0) 0x7fde240725b0 con 0x7fde24155eb0 2026-02-22T23:28:27.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 >> v1:10.20.193.80:6789/0 conn(0x7fde2415c970 legacy=0x7fde240c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 >> v1:10.20.193.6:6789/0 conn(0x7fde2406ff20 legacy=0x7fde241557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: _finish_hunting 0 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde24072e50 con 0x7fde24155eb0 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: _finish_auth 0 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: _check_auth_tickets 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fde0c002ee0 con 0x7fde24155eb0 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:27.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: dump: 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:27.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 >> v1:10.20.193.40:6789/0 conn(0x7fde24155eb0 legacy=0x7fde240c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde10003f90 0x7fde10004370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): picked mon.a con 0x7fde10003f90 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde10004a30 0x7fde10004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): picked mon.b con 0x7fde10004a30 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10009ab0 0x7fde1000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): picked mon.c con 0x7fde10009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fde1000c4e0 con 0x7fde10003f90 2026-02-22T23:28:27.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fde1000c690 con 0x7fde10004a30 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fde1000c890 con 0x7fde10009ab0 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21573640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde21d74640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde10004a30 0x7fde10004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.073+0000 7fde22575640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde10003f90 0x7fde10004370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22575640 10 monclient(hunting): get_auth_request con 0x7fde10003f90 auth_method 0 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde21d74640 10 monclient(hunting): get_auth_request con 0x7fde10004a30 auth_method 0 2026-02-22T23:28:27.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde21d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde21d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde21d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22d76640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10009ab0 0x7fde1000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde22575640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde21d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde21d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.074+0000 7fde21d74640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22d76640 10 monclient(hunting): get_auth_request con 0x7fde10009ab0 auth_method 0 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:27.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22d76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 10 monclient(hunting): handle_auth_done global_id 5056 payload 293 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 1 -- 10.20.193.6:0/2865138381 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10009ab0 msgr2=0x7fde1000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10009ab0 0x7fde1000bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 1 -- 10.20.193.6:0/2865138381 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde10004a30 msgr2=0x7fde10004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde10004a30 0x7fde10004eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 10 monclient: _finish_hunting 0 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 1 monclient: found mon.a 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 1 -- 10.20.193.6:0/2865138381 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde240725b0 con 0x7fde10003f90 2026-02-22T23:28:27.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 10 monclient: _finish_auth 0 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 10 monclient: _check_auth_tickets 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22575640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde28f14640 5 monclient: authenticate success, global_id 5056 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde28f14640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.075+0000 7fde22d76640 1 -- 10.20.193.6:0/2865138381 reap_dead start 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.076+0000 7fde22575640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde10003f90 0x7fde10004370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde18003370 tx=0x7fde1802f060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79fce40dac0b1ed server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18002f90 con 0x7fde10003f90 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 10 monclient: dump: 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:27.091 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18033070 con 0x7fde10003f90 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 1 -- 10.20.193.6:0/2865138381 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18004de0 con 0x7fde10003f90 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:27.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde21573640 10 monclient: dump: 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:27.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 4 set_mon_vals no callback set 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 10 monclient: shutdown 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 1 -- 10.20.193.6:0/2865138381 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde10003f90 msgr2=0x7fde10004370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde10003f90 0x7fde10004370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fde18003370 tx=0x7fde1802f060 comp rx=0 tx=0).stop 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 1 -- 10.20.193.6:0/2865138381 shutdown_connections 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 1 --2- 10.20.193.6:0/2865138381 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde10003f90 0x7fde10004370 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 1 -- 10.20.193.6:0/2865138381 >> 10.20.193.6:0/2865138381 conn(0x7fde240cd780 msgr2=0x7fde240cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 1 -- 10.20.193.6:0/2865138381 shutdown_connections 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.078+0000 7fde28f14640 1 -- 10.20.193.6:0/2865138381 wait complete. 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient: build_initial_monmap 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient: monmap: 2026-02-22T23:28:27.094 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 librados: starting msgr at 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 librados: starting objecter 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 Processor -- start 2026-02-22T23:28:27.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 -- start start 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 librados: setting wanted keys 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 librados: calling monclient init 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient: init 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10003f90 0x7fde24147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): picked mon.noname-c con 0x7fde10003f90 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde241483c0 0x7fde2415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): picked mon.noname-a con 0x7fde241483c0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde2415a640 0x7fde24154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): picked mon.noname-b con 0x7fde2415a640 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:27.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fde24070480 con 0x7fde241483c0 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fde24070300 con 0x7fde2415a640 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fde24070600 con 0x7fde10003f90 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.079+0000 7fde28f14640 10 monclient(hunting): authenticate will time out at 1381.806900s 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde241483c0 0x7fde2415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde2415a640 0x7fde24154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde241483c0 0x7fde2415a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:54328/0 (socket says 10.20.193.6:54328) 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10003f90 0x7fde24147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 1 -- 10.20.193.6:0/4289921193 learned_addr learned my addr 10.20.193.6:0/4289921193 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 10 monclient(hunting): get_auth_request con 0x7fde241483c0 auth_method 0 2026-02-22T23:28:27.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 10 monclient(hunting): get_auth_request con 0x7fde2415a640 auth_method 0 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 10 monclient(hunting): get_auth_request con 0x7fde10003f90 auth_method 0 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:27.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde21d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.080+0000 7fde22575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 10 monclient(hunting): handle_auth_done global_id 5103 payload 824 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 1 -- 10.20.193.6:0/4289921193 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10003f90 msgr2=0x7fde24147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10003f90 0x7fde24147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 1 -- 10.20.193.6:0/4289921193 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde2415a640 msgr2=0x7fde24154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde2415a640 0x7fde24154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 10 monclient: _finish_hunting 0 2026-02-22T23:28:27.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 1 monclient: found mon.noname-a 2026-02-22T23:28:27.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:27.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:27.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.081+0000 7fde21d74640 1 -- 10.20.193.6:0/4289921193 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2415aac0 con 0x7fde241483c0 2026-02-22T23:28:27.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.082+0000 7fde21d74640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde241483c0 0x7fde2415a100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00f8f0 tx=0x7fde0c00fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:27.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.082+0000 7fde0affd640 10 client.?.objecter ms_handle_connect 0x7fde241483c0 2026-02-22T23:28:27.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.082+0000 7fde0affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:27.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 1 -- 10.20.193.6:0/4289921193 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c0052e0 con 0x7fde241483c0 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: dump: 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:27.103 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: _finish_auth 0 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 1 -- 10.20.193.6:0/4289921193 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde0c018070 con 0x7fde241483c0 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 5 monclient: authenticate success, global_id 5103 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 monclient: _renew_subs 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241f3a80 con 0x7fde241483c0 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:27.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 1 -- 10.20.193.6:0/4289921193 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c0036f0 con 0x7fde241483c0 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde20d72640 4 set_mon_vals no callback set 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde0affd640 10 monclient: dump: 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:27.105 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 client.5103.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 monclient: _renew_subs 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241f4040 con 0x7fde241483c0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 1 librados: init done 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.084+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2415e340 con 0x7fde241483c0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.095+0000 7fde0affd640 1 -- 10.20.193.6:0/4289921193 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fde0c003890 con 0x7fde241483c0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.096+0000 7fde0affd640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fddf0078e50 0x7fddf007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.096+0000 7fde22575640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fddf0078e50 0x7fddf007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.096+0000 7fde22575640 10 monclient: get_auth_request con 0x7fddf0078e50 auth_method 0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.097+0000 7fde22575640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fddf0078e50 0x7fddf007b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde18036000 tx=0x7fde18007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:27.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.097+0000 7fde0affd640 10 client.5103.objecter ms_handle_connect 0x7fddf0078e50 2026-02-22T23:28:27.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.097+0000 7fde0affd640 1 -- 10.20.193.6:0/4289921193 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fde0c05e390 con 0x7fde241483c0 2026-02-22T23:28:27.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.097+0000 7fde0affd640 10 client.5103.objecter ms_dispatch2 0x7fde24000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:27.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.097+0000 7fde0affd640 3 client.5103.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:27.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.097+0000 7fde0affd640 3 client.5103.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:27.107 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.097+0000 7fde0affd640 20 client.5103.objecter dump_active .. 0 homeless 2026-02-22T23:28:27.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.194+0000 7fde0affd640 1 -- 10.20.193.6:0/4289921193 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fde2415aac0 con 0x7fde241483c0 2026-02-22T23:28:27.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.194+0000 7fde0affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:27.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.194+0000 7fde0affd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:27.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.293+0000 7fde28f14640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:27.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.293+0000 7fde28f14640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:27.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.293+0000 7fde28f14640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:27.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.293+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fde2415b130 con 0x7fde241483c0 2026-02-22T23:28:27.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.296+0000 7fde0affd640 1 -- 10.20.193.6:0/4289921193 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fde0c062350 con 0x7fde241483c0 2026-02-22T23:28:27.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.296+0000 7fde0affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:27.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.296+0000 7fde0affd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:27.298 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:27.298 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 10 librados: watch_flush enter 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 10 librados: watch_flush exit 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 20 client.5103.objecter shutdown clearing up homeless session... 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 10 client.5103.objecter successfully canceled tick 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fddf0078e50 msgr2=0x7fddf007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fddf0078e50 0x7fddf007b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde18036000 tx=0x7fde18007000 comp rx=0 tx=0).stop 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 10 monclient: shutdown 2026-02-22T23:28:27.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde241483c0 msgr2=0x7fde2415a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde241483c0 0x7fde2415a100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00f8f0 tx=0x7fde0c00fdc0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 shutdown_connections 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fddf0078e50 0x7fddf007b310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fde2415a640 0x7fde24154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fde241483c0 0x7fde2415a100 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 --2- 10.20.193.6:0/4289921193 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fde10003f90 0x7fde24147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 >> 10.20.193.6:0/4289921193 conn(0x7fde240cd780 msgr2=0x7fde24155570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 shutdown_connections 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 -- 10.20.193.6:0/4289921193 wait complete. 2026-02-22T23:28:27.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:27.297+0000 7fde28f14640 1 librados: shutdown 2026-02-22T23:28:30.308 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:30.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.378+0000 7f60e2111640 10 monclient: get_monmap_and_config 2026-02-22T23:28:30.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.378+0000 7f60e2111640 10 monclient: build_initial_monmap 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.378+0000 7f60e2111640 10 monclient: monmap: 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:30.379443+0000 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:30.379443+0000 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:30.381 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:30.382 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:30.382 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:30.382 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:30.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 1 Processor -- start 2026-02-22T23:28:30.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 1 -- start start 2026-02-22T23:28:30.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient: init 2026-02-22T23:28:30.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:30.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient: _add_conns ranks=[2,0,3,4,1,5] 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f60dc06ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f60dc0c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f60dc15a5e0 0x7f60dc070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient(hunting): picked mon.noname-a con 0x7f60dc15a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60dc0c7e90 con 0x7f60dc0c99a0 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60dc0c8170 con 0x7f60dc06ff20 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f60dc159a40 con 0x7f60dc15a5e0 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.379+0000 7f60e2111640 10 monclient(hunting): authenticate will time out at 1385.106805s 2026-02-22T23:28:30.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.380+0000 7f60dbfff640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f60dc15a5e0 0x7f60dc070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.380+0000 7f60db7fe640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f60dc06ff20 0x7f60dc159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:40474/0 (socket says 10.20.193.6:40474) 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.380+0000 7f60db7fe640 1 -- 10.20.193.6:0/3788403016 learned_addr learned my addr 10.20.193.6:0/3788403016 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.380+0000 7f60dbfff640 10 monclient(hunting): get_auth_request con 0x7f60dc15a5e0 auth_method 0 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.380+0000 7f60dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.380+0000 7f60dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.380+0000 7f60dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60da7fc640 1 -- 10.20.193.6:0/3788403016 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709433690 0 0) 0x7f60dc0c8170 con 0x7f60dc06ff20 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60da7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60da7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:30.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60da7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60da7fc640 10 monclient(hunting): my global_id is 5189 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.381+0000 7f60da7fc640 1 -- 10.20.193.6:0/3788403016 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c0003620 con 0x7f60dc06ff20 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 10 monclient(hunting): handle_auth_done global_id 5106 payload 274 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 1 -- 10.20.193.6:0/3788403016 >> v1:10.20.193.80:6789/0 conn(0x7f60dc06ff20 legacy=0x7f60dc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 1 -- 10.20.193.6:0/3788403016 >> v1:10.20.193.6:6789/0 conn(0x7f60dc0c99a0 legacy=0x7f60dc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 1 monclient: found mon.noname-a 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 1 -- 10.20.193.6:0/3788403016 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc0c8450 con 0x7f60dc15a5e0 2026-02-22T23:28:30.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.382+0000 7f60dbfff640 1 --2- 10.20.193.6:0/3788403016 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc15a5e0 0x7f60dc070300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009dc0 tx=0x7f60d002f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933b9d9a10e5076 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 1 -- 10.20.193.6:0/3788403016 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d003c070 con 0x7f60dc15a5e0 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: dump: 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.393 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 1 -- 10.20.193.6:0/3788403016 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d0037440 con 0x7f60dc15a5e0 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 5 monclient: authenticate success, global_id 5106 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:30.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:30.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 1 -- 10.20.193.6:0/3788403016 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0037810 con 0x7f60dc15a5e0 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60da7fc640 10 monclient: dump: 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:30.396 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 4 set_mon_vals no callback set 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 10 monclient: shutdown 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 1 -- 10.20.193.6:0/3788403016 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc15a5e0 msgr2=0x7f60dc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 1 --2- 10.20.193.6:0/3788403016 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc15a5e0 0x7f60dc070300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009dc0 tx=0x7f60d002f230 comp rx=0 tx=0).stop 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 1 -- 10.20.193.6:0/3788403016 shutdown_connections 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 1 --2- 10.20.193.6:0/3788403016 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc15a5e0 0x7f60dc070300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 1 -- 10.20.193.6:0/3788403016 >> 10.20.193.6:0/3788403016 conn(0x7f60dc0cd780 msgr2=0x7f60dc0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:30.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 1 -- 10.20.193.6:0/3788403016 shutdown_connections 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.385+0000 7f60e2111640 1 -- 10.20.193.6:0/3788403016 wait complete. 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 10 monclient: build_initial_monmap 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 10 monclient: monmap: 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:30.398 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 1 librados: starting msgr at 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 1 librados: starting objecter 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 1 Processor -- start 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 1 -- start start 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 1 librados: setting wanted keys 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 1 librados: calling monclient init 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.386+0000 7f60e2111640 10 monclient: init 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60dc06ff20 0x7f60dc1f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:30.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): picked mon.noname-b con 0x7f60dc06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc0c99a0 0x7f60dc1f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): picked mon.noname-a con 0x7f60dc0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60dc15a5e0 0x7f60dc1f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): picked mon.noname-c con 0x7f60dc15a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc150520 con 0x7f60dc0c99a0 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc0c8a10 con 0x7f60dc06ff20 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1506a0 con 0x7f60dc15a5e0 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60e2111640 10 monclient(hunting): authenticate will time out at 1385.114084s 2026-02-22T23:28:30.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc0c99a0 0x7f60dc1f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60dbfff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60dc15a5e0 0x7f60dc1f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc0c99a0 0x7f60dc1f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49806/0 (socket says 10.20.193.6:49806) 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 1 -- 10.20.193.6:0/4095639931 learned_addr learned my addr 10.20.193.6:0/4095639931 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 10 monclient(hunting): get_auth_request con 0x7f60dc0c99a0 auth_method 0 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60db7fe640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60dc06ff20 0x7f60dc1f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60dbfff640 10 monclient(hunting): get_auth_request con 0x7f60dc15a5e0 auth_method 0 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60dbfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:30.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60dbfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60dbfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60db7fe640 10 monclient(hunting): get_auth_request con 0x7f60dc06ff20 auth_method 0 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60db7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60db7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60db7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.387+0000 7f60daffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.388+0000 7f60dbfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.388+0000 7f60dbfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.388+0000 7f60dbfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.388+0000 7f60db7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:30.402 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.388+0000 7f60db7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.388+0000 7f60db7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 10 monclient(hunting): handle_auth_done global_id 5109 payload 824 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 1 -- 10.20.193.6:0/4095639931 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60dc15a5e0 msgr2=0x7f60dc1f2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60dc15a5e0 0x7f60dc1f2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 1 -- 10.20.193.6:0/4095639931 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60dc06ff20 msgr2=0x7f60dc1f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60dc06ff20 0x7f60dc1f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 1 monclient: found mon.noname-a 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.389+0000 7f60daffd640 1 -- 10.20.193.6:0/4095639931 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc1f1470 con 0x7f60dc0c99a0 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.390+0000 7f60daffd640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc0c99a0 0x7f60dc1f2510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00b570 tx=0x7f60cc00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.390+0000 7f60b7fff640 10 client.?.objecter ms_handle_connect 0x7f60dc0c99a0 2026-02-22T23:28:30.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.390+0000 7f60b7fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 1 -- 10.20.193.6:0/4095639931 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc013020 con 0x7f60dc0c99a0 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: dump: 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:30.406 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 1 -- 10.20.193.6:0/4095639931 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60cc0044a0 con 0x7f60dc0c99a0 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 5 monclient: authenticate success, global_id 5109 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 monclient: _renew_subs 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc1f1750 con 0x7f60dc0c99a0 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 1 -- 10.20.193.6:0/4095639931 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0029c0 con 0x7f60dc0c99a0 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:30.407 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60d9ffb640 4 set_mon_vals no callback set 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60b7fff640 10 monclient: dump: 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 client.5109.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 monclient: _renew_subs 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:30.408 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1f1d10 con 0x7f60dc0c99a0 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 1 librados: init done 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.392+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc070300 con 0x7f60dc0c99a0 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.403+0000 7f60b7fff640 1 -- 10.20.193.6:0/4095639931 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f60cc002b60 con 0x7f60dc0c99a0 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.404+0000 7f60b7fff640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60ac078e00 0x7f60ac07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.404+0000 7f60db7fe640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60ac078e00 0x7f60ac07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:30.409 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.404+0000 7f60db7fe640 10 monclient: get_auth_request con 0x7f60ac078e00 auth_method 0 2026-02-22T23:28:30.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.405+0000 7f60db7fe640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60ac078e00 0x7f60ac07b2c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60c8007b60 tx=0x7f60c8007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:30.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.405+0000 7f60b7fff640 10 client.5109.objecter ms_handle_connect 0x7f60ac078e00 2026-02-22T23:28:30.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.406+0000 7f60b7fff640 1 -- 10.20.193.6:0/4095639931 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f60cc05dff0 con 0x7f60dc0c99a0 2026-02-22T23:28:30.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.406+0000 7f60b7fff640 10 client.5109.objecter ms_dispatch2 0x7f60dc000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:30.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.406+0000 7f60b7fff640 3 client.5109.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:30.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.406+0000 7f60b7fff640 3 client.5109.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:30.410 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.406+0000 7f60b7fff640 20 client.5109.objecter dump_active .. 0 homeless 2026-02-22T23:28:30.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.505+0000 7f60b7fff640 1 -- 10.20.193.6:0/4095639931 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f60cc01d020 con 0x7f60dc0c99a0 2026-02-22T23:28:30.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.505+0000 7f60b7fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:30.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.505+0000 7f60b7fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:30.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.605+0000 7f60e2111640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:30.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.605+0000 7f60e2111640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:30.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.605+0000 7f60e2111640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:30.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.605+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f60dc0c49a0 con 0x7f60dc0c99a0 2026-02-22T23:28:30.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.607+0000 7f60b7fff640 1 -- 10.20.193.6:0/4095639931 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7f60cc061fb0 con 0x7f60dc0c99a0 2026-02-22T23:28:30.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.607+0000 7f60b7fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:30.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.607+0000 7f60b7fff640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:30.609 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:30.609 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 10 librados: watch_flush enter 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 10 librados: watch_flush exit 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 20 client.5109.objecter shutdown clearing up homeless session... 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 10 client.5109.objecter successfully canceled tick 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60ac078e00 msgr2=0x7f60ac07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60ac078e00 0x7f60ac07b2c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60c8007b60 tx=0x7f60c8007ab0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 10 monclient: shutdown 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:30.611 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc0c99a0 msgr2=0x7f60dc1f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc0c99a0 0x7f60dc1f2510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00b570 tx=0x7f60cc00ba40 comp rx=0 tx=0).stop 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 shutdown_connections 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f60ac078e00 0x7f60ac07b2c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f60dc15a5e0 0x7f60dc1f2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f60dc0c99a0 0x7f60dc1f2510 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 --2- 10.20.193.6:0/4095639931 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f60dc06ff20 0x7f60dc1f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 >> 10.20.193.6:0/4095639931 conn(0x7f60dc0cd780 msgr2=0x7f60dc073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 shutdown_connections 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 -- 10.20.193.6:0/4095639931 wait complete. 2026-02-22T23:28:30.612 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:30.608+0000 7f60e2111640 1 librados: shutdown 2026-02-22T23:28:33.405 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:17.655 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.407 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:17.657 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.414 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:17.664 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.434 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:17.645 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.435 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:17.682 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.436 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:17.647 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.437 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:17.683 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.444 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:17.655 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.445 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:17.691 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:33.619 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.687+0000 7fb85a78f640 10 monclient: get_monmap_and_config 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.687+0000 7fb85a78f640 10 monclient: build_initial_monmap 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.687+0000 7fb85a78f640 10 monclient: monmap: 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:33.688281+0000 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:33.688281+0000 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:33.689 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:33.690 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:33.690 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:33.690 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:33.690 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:33.690 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:33.690 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:33.690 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 1 Processor -- start 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 1 -- start start 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient: init 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient: _add_conns ranks=[0,2,5,3,1,4] 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb8541527d0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb854152bb0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fb8540c6190 0x7fb8540c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient(hunting): picked mon.noname-c con 0x7fb8540c6190 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85406e300 con 0x7fb8541527d0 2026-02-22T23:28:33.693 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85406e5e0 con 0x7fb854152bb0 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb85406f1f0 con 0x7fb8540c6190 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.688+0000 7fb85a78f640 10 monclient(hunting): authenticate will time out at 1388.415743s 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.689+0000 7fb853fff640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fb8541527d0 0x7fb85406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35172/0 (socket says 10.20.193.6:35172) 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.689+0000 7fb853fff640 1 -- 10.20.193.6:0/1769618824 learned_addr learned my addr 10.20.193.6:0/1769618824 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.689+0000 7fb858d05640 1 --2- 10.20.193.6:0/1769618824 >> v2:10.20.193.80:3300/0 conn(0x7fb8540c6190 0x7fb8540c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.689+0000 7fb858d05640 10 monclient(hunting): get_auth_request con 0x7fb8540c6190 auth_method 0 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.689+0000 7fb858d05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.689+0000 7fb858d05640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.689+0000 7fb858d05640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:33.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb858d05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb858d05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb858d05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb852ffd640 1 -- 10.20.193.6:0/1769618824 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 796404486 0 0) 0x7fb85406e300 con 0x7fb8541527d0 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb852ffd640 10 monclient(hunting): _negotiate 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb852ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb852ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb852ffd640 10 monclient(hunting): my global_id is 5112 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.690+0000 7fb852ffd640 1 -- 10.20.193.6:0/1769618824 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8380058b0 con 0x7fb8541527d0 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 10 monclient(hunting): handle_auth_done global_id 5195 payload 274 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 1 -- 10.20.193.6:0/1769618824 >> v1:10.20.193.80:6789/0 conn(0x7fb854152bb0 legacy=0x7fb8540c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 1 -- 10.20.193.6:0/1769618824 >> v1:10.20.193.6:6789/0 conn(0x7fb8541527d0 legacy=0x7fb85406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:33.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 10 monclient: _finish_hunting 0 2026-02-22T23:28:33.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 1 monclient: found mon.noname-c 2026-02-22T23:28:33.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:33.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:33.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.691+0000 7fb858d05640 1 -- 10.20.193.6:0/1769618824 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85406e8c0 con 0x7fb8540c6190 2026-02-22T23:28:33.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.693+0000 7fb858d05640 1 --2- 10.20.193.6:0/1769618824 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb8540c8e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb85406f370 tx=0x7fb84802f230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=391bc03eb5668d17 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 1 -- 10.20.193.6:0/1769618824 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84803c070 con 0x7fb8540c6190 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: dump: 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.701 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 1 -- 10.20.193.6:0/1769618824 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb848037440 con 0x7fb8540c6190 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb85a78f640 5 monclient: authenticate success, global_id 5195 2026-02-22T23:28:33.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb85a78f640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb85a78f640 4 set_mon_vals no callback set 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 1 -- 10.20.193.6:0/1769618824 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848037810 con 0x7fb8540c6190 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.695+0000 7fb852ffd640 10 monclient: dump: 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:33.703 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 10 monclient: shutdown 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 -- 10.20.193.6:0/1769618824 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 msgr2=0x7fb8540c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 --2- 10.20.193.6:0/1769618824 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb8540c8e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb85406f370 tx=0x7fb84802f230 comp rx=0 tx=0).stop 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 -- 10.20.193.6:0/1769618824 shutdown_connections 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 --2- 10.20.193.6:0/1769618824 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb8540c8e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 -- 10.20.193.6:0/1769618824 >> 10.20.193.6:0/1769618824 conn(0x7fb8540cd780 msgr2=0x7fb8540cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 -- 10.20.193.6:0/1769618824 shutdown_connections 2026-02-22T23:28:33.704 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 -- 10.20.193.6:0/1769618824 wait complete. 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 10 monclient: build_initial_monmap 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 10 monclient: monmap: 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:33.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 librados: starting msgr at 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 librados: starting objecter 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.696+0000 7fb85a78f640 1 Processor -- start 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 -- start start 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 librados: setting wanted keys 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 librados: calling monclient init 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient: init 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb85414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): picked mon.noname-c con 0x7fb8540c6190 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb8541527d0 0x7fb8540cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): picked mon.noname-a con 0x7fb8541527d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:33.706 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb854152bb0 0x7fb8540cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): picked mon.noname-b con 0x7fb854152bb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb8540c93a0 con 0x7fb8541527d0 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb85406ee80 con 0x7fb854152bb0 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb8540c9520 con 0x7fb8540c6190 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb85a78f640 10 monclient(hunting): authenticate will time out at 1388.424466s 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb8537fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb8541527d0 0x7fb8540cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.697+0000 7fb853fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb85414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb8541527d0 0x7fb8540cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49810/0 (socket says 10.20.193.6:49810) 2026-02-22T23:28:33.707 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 1 -- 10.20.193.6:0/819648431 learned_addr learned my addr 10.20.193.6:0/819648431 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 10 monclient(hunting): get_auth_request con 0x7fb8541527d0 auth_method 0 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb853fff640 10 monclient(hunting): get_auth_request con 0x7fb8540c6190 auth_method 0 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb853fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb853fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb853fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb854152bb0 0x7fb8540cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:33.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb8537fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb853fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb853fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb853fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 10 monclient(hunting): get_auth_request con 0x7fb854152bb0 auth_method 0 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.698+0000 7fb858d05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 10 monclient(hunting): handle_auth_done global_id 5201 payload 824 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 1 -- 10.20.193.6:0/819648431 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb854152bb0 msgr2=0x7fb8540cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:33.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb854152bb0 0x7fb8540cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 1 -- 10.20.193.6:0/819648431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb8541527d0 msgr2=0x7fb8540cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:33.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb8541527d0 0x7fb8540cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:33.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 1 monclient: found mon.noname-c 2026-02-22T23:28:33.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:33.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:33.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.699+0000 7fb853fff640 1 -- 10.20.193.6:0/819648431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85414dd30 con 0x7fb8540c6190 2026-02-22T23:28:33.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.700+0000 7fb853fff640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb85414f0b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb84000c4d0 tx=0x7fb84000c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:33.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.700+0000 7fb850ff9640 10 client.?.objecter ms_handle_connect 0x7fb8540c6190 2026-02-22T23:28:33.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.700+0000 7fb850ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:33.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb850ff9640 1 -- 10.20.193.6:0/819648431 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840017070 con 0x7fb8540c6190 2026-02-22T23:28:33.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb850ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:33.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb850ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb850ff9640 10 monclient: dump: 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb850ff9640 10 monclient: _finish_auth 0 2026-02-22T23:28:33.713 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb850ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb850ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 5 monclient: authenticate success, global_id 5201 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 10 monclient: _renew_subs 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85414e010 con 0x7fb8540c6190 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 10 client.5201.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 10 monclient: _renew_subs 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85414e5d0 con 0x7fb8540c6190 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.701+0000 7fb85a78f640 1 librados: init done 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb85a78f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:33.714 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb85a78f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb85a78f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8540c5a80 con 0x7fb8540c6190 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb850ff9640 1 -- 10.20.193.6:0/819648431 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb840005de0 con 0x7fb8540c6190 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb850ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb850ff9640 1 -- 10.20.193.6:0/819648431 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840021060 con 0x7fb8540c6190 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb850ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb850ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb850ff9640 10 monclient: dump: 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:33.715 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:33.716 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:33.716 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:33.716 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:33.716 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:33.716 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:33.716 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.702+0000 7fb8527fc640 4 set_mon_vals no callback set 2026-02-22T23:28:33.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.808+0000 7fb850ff9640 1 -- 10.20.193.6:0/819648431 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb840024080 con 0x7fb8540c6190 2026-02-22T23:28:33.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.809+0000 7fb850ff9640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb824078e00 0x7fb82407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:33.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb8537fe640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb824078e00 0x7fb82407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:33.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb8537fe640 10 monclient: get_auth_request con 0x7fb824078e00 auth_method 0 2026-02-22T23:28:33.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb850ff9640 1 -- 10.20.193.6:0/819648431 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fb84005d490 con 0x7fb8540c6190 2026-02-22T23:28:33.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb850ff9640 10 client.5201.objecter ms_dispatch2 0x7fb854000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:33.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb850ff9640 3 client.5201.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:33.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb850ff9640 3 client.5201.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:33.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb8537fe640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb824078e00 0x7fb82407b2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb85414dbb0 tx=0x7fb844002990 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:33.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb850ff9640 20 client.5201.objecter dump_active .. 0 homeless 2026-02-22T23:28:33.813 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.810+0000 7fb850ff9640 10 client.5201.objecter ms_handle_connect 0x7fb824078e00 2026-02-22T23:28:33.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.823+0000 7fb850ff9640 1 -- 10.20.193.6:0/819648431 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb840061450 con 0x7fb8540c6190 2026-02-22T23:28:33.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.823+0000 7fb850ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:33.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.823+0000 7fb850ff9640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:33.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.925+0000 7fb85a78f640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2026-02-22T23:28:33.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.925+0000 7fb85a78f640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:33.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.925+0000 7fb85a78f640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:33.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.925+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb85414fba0 con 0x7fb8540c6190 2026-02-22T23:28:33.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb850ff9640 1 -- 10.20.193.6:0/819648431 <== mon.2 v2:10.20.193.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+201 (secure 0 0 0) 0x7fb840065750 con 0x7fb8540c6190 2026-02-22T23:28:33.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb850ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2026-02-22T23:28:33.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb850ff9640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:33.929 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:33.929 INFO:teuthology.orchestra.run.trial006.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"14 OSD(s) experiencing slow operations in BlueStore","count":14},"muted":false}},"mutes":[]} 2026-02-22T23:28:33.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 10 librados: watch_flush enter 2026-02-22T23:28:33.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 10 librados: watch_flush exit 2026-02-22T23:28:33.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 20 client.5201.objecter shutdown clearing up homeless session... 2026-02-22T23:28:33.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 10 client.5201.objecter successfully canceled tick 2026-02-22T23:28:33.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb824078e00 msgr2=0x7fb82407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb824078e00 0x7fb82407b2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb85414dbb0 tx=0x7fb844002990 comp rx=0 tx=0).stop 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 10 monclient: shutdown 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 msgr2=0x7fb85414f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.928+0000 7fb85a78f640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb85414f0b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb84000c4d0 tx=0x7fb84000c9a0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 shutdown_connections 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb824078e00 0x7fb82407b2c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb854152bb0 0x7fb8540cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb8541527d0 0x7fb8540cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 --2- 10.20.193.6:0/819648431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb8540c6190 0x7fb85414f0b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 >> 10.20.193.6:0/819648431 conn(0x7fb8540cd780 msgr2=0x7fb8540c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:33.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 shutdown_connections 2026-02-22T23:28:33.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 -- 10.20.193.6:0/819648431 wait complete. 2026-02-22T23:28:33.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:33.929+0000 7fb85a78f640 1 librados: shutdown 2026-02-22T23:28:33.939 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:28:33.939 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:28:34.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 10 monclient: get_monmap_and_config 2026-02-22T23:28:34.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 10 monclient: monmap: 2026-02-22T23:28:34.007 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:34.006691+0000 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:34.006691+0000 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:34.008 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 1 Processor -- start 2026-02-22T23:28:34.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 1 -- start start 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 10 monclient: init 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.006+0000 7fef8bcea640 10 monclient: _add_conns ranks=[5,2,4,0,1,3] 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fef8406ff20 0x7fef84070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 10 monclient(hunting): picked mon.noname-c con 0x7fef8406ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fef840c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fef8415a5e0 0x7fef8414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 10 monclient(hunting): picked mon.noname-b con 0x7fef8415a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef840c7e90 con 0x7fef840c99a0 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fef84159a40 con 0x7fef8415a5e0 2026-02-22T23:28:34.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fef840c8170 con 0x7fef8406ff20 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8bcea640 10 monclient(hunting): authenticate will time out at 1388.734051s 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef89a5f640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fef8406ff20 0x7fef84070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8a260640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fef8415a5e0 0x7fef8414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8a260640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fef8415a5e0 0x7fef8414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:59856/0 (socket says 10.20.193.6:59856) 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8a260640 1 -- 10.20.193.6:0/1358159695 learned_addr learned my addr 10.20.193.6:0/1358159695 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8a260640 10 monclient(hunting): get_auth_request con 0x7fef8415a5e0 auth_method 0 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8a260640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8a260640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef8a260640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef89a5f640 10 monclient(hunting): get_auth_request con 0x7fef8406ff20 auth_method 0 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef89a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef89a5f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.007+0000 7fef89a5f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef89a5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef89a5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef89a5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 10 monclient(hunting): handle_auth_done global_id 5068 payload 274 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 1 -- 10.20.193.6:0/1358159695 >> v2:10.20.193.80:3300/0 conn(0x7fef8406ff20 msgr2=0x7fef84070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 1 --2- 10.20.193.6:0/1358159695 >> v2:10.20.193.80:3300/0 conn(0x7fef8406ff20 0x7fef84070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 1 -- 10.20.193.6:0/1358159695 >> v1:10.20.193.80:6789/0 conn(0x7fef840c99a0 legacy=0x7fef84159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 1 monclient: found mon.noname-b 2026-02-22T23:28:34.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.008+0000 7fef8a260640 1 -- 10.20.193.6:0/1358159695 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef840c8450 con 0x7fef8415a5e0 2026-02-22T23:28:34.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.010+0000 7fef8a260640 1 --2- 10.20.193.6:0/1358159695 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef8414fe10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fef80009e50 tx=0x7fef8002f2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a69c049bff7d023 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 1 -- 10.20.193.6:0/1358159695 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8002fd60 con 0x7fef8415a5e0 2026-02-22T23:28:34.021 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: dump: 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.022 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 1 -- 10.20.193.6:0/1358159695 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef800373f0 con 0x7fef8415a5e0 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef8bcea640 5 monclient: authenticate success, global_id 5068 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 1 -- 10.20.193.6:0/1358159695 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800377c0 con 0x7fef8415a5e0 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef8bcea640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef8bcea640 4 set_mon_vals no callback set 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:34.023 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.011+0000 7fef88a5d640 10 monclient: dump: 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 10 monclient: shutdown 2026-02-22T23:28:34.024 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 -- 10.20.193.6:0/1358159695 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 msgr2=0x7fef8414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 --2- 10.20.193.6:0/1358159695 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef8414fe10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fef80009e50 tx=0x7fef8002f2a0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 -- 10.20.193.6:0/1358159695 shutdown_connections 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 --2- 10.20.193.6:0/1358159695 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef8414fe10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 --2- 10.20.193.6:0/1358159695 >> v2:10.20.193.80:3300/0 conn(0x7fef8406ff20 0x7fef84070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 -- 10.20.193.6:0/1358159695 >> 10.20.193.6:0/1358159695 conn(0x7fef840cd780 msgr2=0x7fef840cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 -- 10.20.193.6:0/1358159695 shutdown_connections 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 1 -- 10.20.193.6:0/1358159695 wait complete. 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.012+0000 7fef8bcea640 10 monclient: monmap: 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.025 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 librados: starting msgr at 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 librados: starting objecter 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 Processor -- start 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 -- start start 2026-02-22T23:28:34.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 librados: setting wanted keys 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 librados: calling monclient init 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient: init 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef8406ff20 0x7fef8406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): picked mon.noname-a con 0x7fef8406ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef840c99a0 0x7fef8406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): picked mon.noname-c con 0x7fef840c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef841f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): picked mon.noname-b con 0x7fef8415a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fef84150350 con 0x7fef8406ff20 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.027 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fef840c8a10 con 0x7fef8415a5e0 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef841504d0 con 0x7fef840c99a0 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8bcea640 10 monclient(hunting): authenticate will time out at 1388.740653s 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8925e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef840c99a0 0x7fef8406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.013+0000 7fef8a260640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef841f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef8406ff20 0x7fef8406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef840c99a0 0x7fef8406e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:57028/0 (socket says 10.20.193.6:57028) 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 1 -- 10.20.193.6:0/3052803519 learned_addr learned my addr 10.20.193.6:0/3052803519 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 10 monclient(hunting): get_auth_request con 0x7fef840c99a0 auth_method 0 2026-02-22T23:28:34.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8a260640 10 monclient(hunting): get_auth_request con 0x7fef8415a5e0 auth_method 0 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8a260640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8a260640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8a260640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 10 monclient(hunting): get_auth_request con 0x7fef8406ff20 auth_method 0 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8a260640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8a260640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8a260640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef8925e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.014+0000 7fef89a5f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 10 monclient(hunting): handle_auth_done global_id 5071 payload 824 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 1 -- 10.20.193.6:0/3052803519 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef840c99a0 msgr2=0x7fef8406e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef840c99a0 0x7fef8406e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 1 -- 10.20.193.6:0/3052803519 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef8406ff20 msgr2=0x7fef8406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef8406ff20 0x7fef8406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 1 monclient: found mon.noname-b 2026-02-22T23:28:34.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.015+0000 7fef8a260640 1 -- 10.20.193.6:0/3052803519 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef840c3010 con 0x7fef8415a5e0 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.016+0000 7fef8a260640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef841f72b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef80035000 tx=0x7fef80004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.016+0000 7fef727fc640 10 client.?.objecter ms_handle_connect 0x7fef8415a5e0 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.016+0000 7fef727fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 1 -- 10.20.193.6:0/3052803519 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80046020 con 0x7fef8415a5e0 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: dump: 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.031 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.032 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 1 -- 10.20.193.6:0/3052803519 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80041450 con 0x7fef8415a5e0 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 1 -- 10.20.193.6:0/3052803519 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80009510 con 0x7fef8415a5e0 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef727fc640 10 monclient: dump: 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.033 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef73fff640 4 set_mon_vals no callback set 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 5 monclient: authenticate success, global_id 5071 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 10 monclient: _renew_subs 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef840c32b0 con 0x7fef8415a5e0 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 10 client.5071.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 10 monclient: _renew_subs 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef840c3870 con 0x7fef8415a5e0 2026-02-22T23:28:34.034 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.018+0000 7fef8bcea640 1 librados: init done 2026-02-22T23:28:34.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.019+0000 7fef8bcea640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.019+0000 7fef8bcea640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.019+0000 7fef8bcea640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.035 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.019+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef84150860 con 0x7fef8415a5e0 2026-02-22T23:28:34.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.125+0000 7fef727fc640 1 -- 10.20.193.6:0/3052803519 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fef80041840 con 0x7fef8415a5e0 2026-02-22T23:28:34.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.125+0000 7fef727fc640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef54078e50 0x7fef5407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.126+0000 7fef89a5f640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef54078e50 0x7fef5407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.126+0000 7fef89a5f640 10 monclient: get_auth_request con 0x7fef54078e50 auth_method 0 2026-02-22T23:28:34.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.126+0000 7fef89a5f640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef54078e50 0x7fef5407b310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef74009c40 tx=0x7fef74009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.126+0000 7fef727fc640 10 client.5071.objecter ms_handle_connect 0x7fef54078e50 2026-02-22T23:28:34.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.127+0000 7fef727fc640 1 -- 10.20.193.6:0/3052803519 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fef800837c0 con 0x7fef8415a5e0 2026-02-22T23:28:34.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.127+0000 7fef727fc640 10 client.5071.objecter ms_dispatch2 0x7fef84000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:34.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.127+0000 7fef727fc640 3 client.5071.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:34.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.127+0000 7fef727fc640 3 client.5071.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:34.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.127+0000 7fef727fc640 20 client.5071.objecter dump_active .. 0 homeless 2026-02-22T23:28:34.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.134+0000 7fef727fc640 1 -- 10.20.193.6:0/3052803519 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fef80050030 con 0x7fef8415a5e0 2026-02-22T23:28:34.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.134+0000 7fef727fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.134+0000 7fef727fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:34.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.215+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef840c48e0 con 0x7fef54078e50 2026-02-22T23:28:34.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.216+0000 7fef727fc640 1 -- 10.20.193.6:0/3052803519 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45281 (secure 0 0 0) 0x7fef840c48e0 con 0x7fef54078e50 2026-02-22T23:28:34.218 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 10 librados: watch_flush enter 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 10 librados: watch_flush exit 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 20 client.5071.objecter shutdown clearing up homeless session... 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 10 client.5071.objecter successfully canceled tick 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef54078e50 msgr2=0x7fef5407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef54078e50 0x7fef5407b310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef74009c40 tx=0x7fef74009290 comp rx=0 tx=0).stop 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 10 monclient: shutdown 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 msgr2=0x7fef841f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef841f72b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef80035000 tx=0x7fef80004b40 comp rx=0 tx=0).stop 2026-02-22T23:28:34.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 shutdown_connections 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fef54078e50 0x7fef5407b310 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fef8415a5e0 0x7fef841f72b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fef840c99a0 0x7fef8406e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 --2- 10.20.193.6:0/3052803519 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fef8406ff20 0x7fef8406ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 >> 10.20.193.6:0/3052803519 conn(0x7fef840cd780 msgr2=0x7fef84159210 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 shutdown_connections 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 -- 10.20.193.6:0/3052803519 wait complete. 2026-02-22T23:28:34.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.217+0000 7fef8bcea640 1 librados: shutdown 2026-02-22T23:28:34.226 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-22T23:28:33.807542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437144,"kb_used_data":5592,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512296,"statfs":{"total":1546188226560,"available":1545740591104,"internally_reserved":0,"allocated":5726208,"data_stored":2779510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25502,"internal_metadata":441424994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002278"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:36.952185+0000","last_change":"2026-02-22T23:23:26.541669+0000","last_active":"2026-02-22T23:25:36.952185+0000","last_peered":"2026-02-22T23:25:36.952185+0000","last_clean":"2026-02-22T23:25:36.952185+0000","last_became_active":"2026-02-22T23:23:24.999148+0000","last_became_peered":"2026-02-22T23:23:24.999148+0000","last_unstale":"2026-02-22T23:25:36.952185+0000","last_undegraded":"2026-02-22T23:25:36.952185+0000","last_fullsized":"2026-02-22T23:25:36.952185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:33:54.198723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033743088999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:34.970590+0000","last_change":"2026-02-22T23:23:26.948831+0000","last_active":"2026-02-22T23:25:34.970590+0000","last_peered":"2026-02-22T23:25:34.970590+0000","last_clean":"2026-02-22T23:25:34.970590+0000","last_became_active":"2026-02-22T23:23:24.966693+0000","last_became_peered":"2026-02-22T23:23:24.966693+0000","last_unstale":"2026-02-22T23:25:34.970590+0000","last_undegraded":"2026-02-22T23:25:34.970590+0000","last_fullsized":"2026-02-22T23:25:34.970590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:30:16.311433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033152011000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:30.040017+0000","last_change":"2026-02-22T23:23:26.535885+0000","last_active":"2026-02-22T23:25:30.040017+0000","last_peered":"2026-02-22T23:25:30.040017+0000","last_clean":"2026-02-22T23:25:30.040017+0000","last_became_active":"2026-02-22T23:23:24.806260+0000","last_became_peered":"2026-02-22T23:23:24.806260+0000","last_unstale":"2026-02-22T23:25:30.040017+0000","last_undegraded":"2026-02-22T23:25:30.040017+0000","last_fullsized":"2026-02-22T23:25:30.040017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:46:46.747409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032290068999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:30.011204+0000","last_change":"2026-02-22T23:23:26.547663+0000","last_active":"2026-02-22T23:25:30.011204+0000","last_peered":"2026-02-22T23:25:30.011204+0000","last_clean":"2026-02-22T23:25:30.011204+0000","last_became_active":"2026-02-22T23:23:24.706162+0000","last_became_peered":"2026-02-22T23:23:24.706162+0000","last_unstale":"2026-02-22T23:25:30.011204+0000","last_undegraded":"2026-02-22T23:25:30.011204+0000","last_fullsized":"2026-02-22T23:25:30.011204+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:37:21.100237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041859805999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"27'2","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.979297+0000","last_change":"2026-02-22T23:23:26.386245+0000","last_active":"2026-02-22T23:25:31.979297+0000","last_peered":"2026-02-22T23:25:31.979297+0000","last_clean":"2026-02-22T23:25:31.979297+0000","last_became_active":"2026-02-22T23:23:25.054022+0000","last_became_peered":"2026-02-22T23:23:25.054022+0000","last_unstale":"2026-02-22T23:25:31.979297+0000","last_undegraded":"2026-02-22T23:25:31.979297+0000","last_fullsized":"2026-02-22T23:25:31.979297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:02:19.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.070943361999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.952620+0000","last_change":"2026-02-22T23:23:26.542103+0000","last_active":"2026-02-22T23:25:31.952620+0000","last_peered":"2026-02-22T23:25:31.952620+0000","last_clean":"2026-02-22T23:25:31.952620+0000","last_became_active":"2026-02-22T23:23:24.462347+0000","last_became_peered":"2026-02-22T23:23:24.462347+0000","last_unstale":"2026-02-22T23:25:31.952620+0000","last_undegraded":"2026-02-22T23:25:31.952620+0000","last_fullsized":"2026-02-22T23:25:31.952620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:09:34.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033702211000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:26.977309+0000","last_change":"2026-02-22T23:23:26.382023+0000","last_active":"2026-02-22T23:25:26.977309+0000","last_peered":"2026-02-22T23:25:26.977309+0000","last_clean":"2026-02-22T23:25:26.977309+0000","last_became_active":"2026-02-22T23:23:24.464964+0000","last_became_peered":"2026-02-22T23:23:24.464964+0000","last_unstale":"2026-02-22T23:25:26.977309+0000","last_undegraded":"2026-02-22T23:25:26.977309+0000","last_fullsized":"2026-02-22T23:25:26.977309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:28:59.922786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055238717,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'1","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.988710+0000","last_change":"2026-02-22T23:23:26.538271+0000","last_active":"2026-02-22T23:25:31.988710+0000","last_peered":"2026-02-22T23:25:31.988710+0000","last_clean":"2026-02-22T23:25:31.988710+0000","last_became_active":"2026-02-22T23:23:24.503509+0000","last_became_peered":"2026-02-22T23:23:24.503509+0000","last_unstale":"2026-02-22T23:25:31.988710+0000","last_undegraded":"2026-02-22T23:25:31.988710+0000","last_fullsized":"2026-02-22T23:25:31.988710+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:47:22.484108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032869134000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"27'32","reported_seq":66,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:21.948881+0000","last_change":"2026-02-22T23:23:22.760169+0000","last_active":"2026-02-22T23:25:21.948881+0000","last_peered":"2026-02-22T23:25:21.948881+0000","last_clean":"2026-02-22T23:25:21.948881+0000","last_became_active":"2026-02-22T23:23:22.740754+0000","last_became_peered":"2026-02-22T23:23:22.740754+0000","last_unstale":"2026-02-22T23:25:21.948881+0000","last_undegraded":"2026-02-22T23:25:21.948881+0000","last_fullsized":"2026-02-22T23:25:21.948881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_clean_scrub_stamp":"2026-02-22T23:23:20.759065+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:14:46.099842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,9,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":15,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2,4,5,6,7,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":22,"seq":94489280578,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,3,4,5,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":6,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,4,5,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":21,"seq":90194313281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[2,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":21,"seq":90194313281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[0,4,6,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280577,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,5,6,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[2,4,5,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,5,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":22,"seq":94489280577,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":22,"seq":94489280578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:28:34.227 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:28:34.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.293+0000 7f060774b640 10 monclient: get_monmap_and_config 2026-02-22T23:28:34.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.293+0000 7f060774b640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.293+0000 7f060774b640 10 monclient: monmap: 2026-02-22T23:28:34.294 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.294 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:34.294207+0000 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:34.294207+0000 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.293+0000 7f060774b640 1 Processor -- start 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 1 -- start start 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient: init 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient: _add_conns ranks=[0,2,4,3,5,1] 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f060006ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f06000c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f060015a5e0 0x7f0600070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient(hunting): picked mon.noname-b con 0x7f060015a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06000c7e90 con 0x7f060006ff20 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06000c8170 con 0x7f06000c99a0 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f0600159a40 con 0x7f060015a5e0 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.294+0000 7f060774b640 10 monclient(hunting): authenticate will time out at 1389.021835s 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f06054c0640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f060006ff20 0x7f0600159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35188/0 (socket says 10.20.193.6:35188) 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f06054c0640 1 -- 10.20.193.6:0/2302353770 learned_addr learned my addr 10.20.193.6:0/2302353770 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 1 --2- 10.20.193.6:0/2302353770 >> v2:10.20.193.40:3300/0 conn(0x7f060015a5e0 0x7f0600070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 10 monclient(hunting): get_auth_request con 0x7f060015a5e0 auth_method 0 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.295+0000 7f0605cc1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 1 -- 10.20.193.6:0/2302353770 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1572548973 0 0) 0x7f06000c7e90 con 0x7f060006ff20 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): my global_id is 5121 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 1 -- 10.20.193.6:0/2302353770 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05e8003620 con 0x7f060006ff20 2026-02-22T23:28:34.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 1 -- 10.20.193.6:0/2302353770 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603770730 0 0) 0x7f06000c8170 con 0x7f06000c99a0 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 10 monclient(hunting): my global_id is 5210 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.296+0000 7f05effff640 1 -- 10.20.193.6:0/2302353770 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06000c7e90 con 0x7f06000c99a0 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 10 monclient(hunting): handle_auth_done global_id 5074 payload 274 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 1 -- 10.20.193.6:0/2302353770 >> v1:10.20.193.80:6789/0 conn(0x7f06000c99a0 legacy=0x7f060014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 1 -- 10.20.193.6:0/2302353770 >> v1:10.20.193.6:6789/0 conn(0x7f060006ff20 legacy=0x7f0600159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 1 monclient: found mon.noname-b 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.297+0000 7f0605cc1640 1 -- 10.20.193.6:0/2302353770 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06000c8450 con 0x7f060015a5e0 2026-02-22T23:28:34.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.298+0000 7f0605cc1640 1 --2- 10.20.193.6:0/2302353770 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600070300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009dc0 tx=0x7f05fc02f280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892286c14ef20eff server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 1 -- 10.20.193.6:0/2302353770 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc03c070 con 0x7f060015a5e0 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: dump: 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 5 monclient: authenticate success, global_id 5074 2026-02-22T23:28:34.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 1 -- 10.20.193.6:0/2302353770 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc037440 con 0x7f060015a5e0 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 1 -- 10.20.193.6:0/2302353770 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0377f0 con 0x7f060015a5e0 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f05effff640 10 monclient: dump: 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 4 set_mon_vals no callback set 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 10 monclient: shutdown 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 1 -- 10.20.193.6:0/2302353770 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 msgr2=0x7f0600070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 1 --2- 10.20.193.6:0/2302353770 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600070300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009dc0 tx=0x7f05fc02f280 comp rx=0 tx=0).stop 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 1 -- 10.20.193.6:0/2302353770 shutdown_connections 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 1 --2- 10.20.193.6:0/2302353770 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600070300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 1 -- 10.20.193.6:0/2302353770 >> 10.20.193.6:0/2302353770 conn(0x7f06000cd780 msgr2=0x7f06000cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 1 -- 10.20.193.6:0/2302353770 shutdown_connections 2026-02-22T23:28:34.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.300+0000 7f060774b640 1 -- 10.20.193.6:0/2302353770 wait complete. 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient: monmap: 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 librados: starting msgr at 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 librados: starting objecter 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 Processor -- start 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 -- start start 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 librados: setting wanted keys 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 librados: calling monclient init 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient: init 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f060006ff20 0x7f060014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): picked mon.noname-c con 0x7f060006ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f06000c99a0 0x7f060006ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): picked mon.noname-a con 0x7f06000c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): picked mon.noname-b con 0x7f060015a5e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0600150520 con 0x7f06000c99a0 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06000c8a10 con 0x7f060015a5e0 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f06001506a0 con 0x7f060006ff20 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.301+0000 7f060774b640 10 monclient(hunting): authenticate will time out at 1389.028964s 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f06000c99a0 0x7f060006ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f060006ff20 0x7f060014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f06000c99a0 0x7f060006ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49826/0 (socket says 10.20.193.6:49826) 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 1 -- 10.20.193.6:0/4077131225 learned_addr learned my addr 10.20.193.6:0/4077131225 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 10 monclient(hunting): get_auth_request con 0x7f06000c99a0 auth_method 0 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 10 monclient(hunting): get_auth_request con 0x7f060015a5e0 auth_method 0 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 10 monclient(hunting): get_auth_request con 0x7f060006ff20 auth_method 0 2026-02-22T23:28:34.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0604cbf640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f0605cc1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.302+0000 7f06054c0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 10 monclient(hunting): handle_auth_done global_id 5077 payload 824 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 1 -- 10.20.193.6:0/4077131225 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f060006ff20 msgr2=0x7f060014cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f060006ff20 0x7f060014cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 1 -- 10.20.193.6:0/4077131225 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f06000c99a0 msgr2=0x7f060006ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f06000c99a0 0x7f060006ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 1 monclient: found mon.noname-b 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.304+0000 7f0605cc1640 1 -- 10.20.193.6:0/4077131225 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0600152250 con 0x7f060015a5e0 2026-02-22T23:28:34.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.305+0000 7f0605cc1640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600152d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05fc036000 tx=0x7f05fc004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.305+0000 7f05ee7fc640 10 client.?.objecter ms_handle_connect 0x7f060015a5e0 2026-02-22T23:28:34.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.305+0000 7f05ee7fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 1 -- 10.20.193.6:0/4077131225 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc03c050 con 0x7f060015a5e0 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: dump: 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 5 monclient: authenticate success, global_id 5077 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 10 monclient: _renew_subs 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0600151970 con 0x7f060015a5e0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 10 client.5077.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 10 monclient: _renew_subs 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0600152510 con 0x7f060015a5e0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 1 librados: init done 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 1 -- 10.20.193.6:0/4077131225 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc048070 con 0x7f060015a5e0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 1 -- 10.20.193.6:0/4077131225 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc009450 con 0x7f060015a5e0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05e75ff640 4 set_mon_vals no callback set 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f05ee7fc640 10 monclient: dump: 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.308 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.306+0000 7f060774b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.307+0000 7f060774b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.307+0000 7f060774b640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:34.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.307+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0600070300 con 0x7f060015a5e0 2026-02-22T23:28:34.412 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.411+0000 7f05ee7fc640 1 -- 10.20.193.6:0/4077131225 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f05fc004040 con 0x7f060015a5e0 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f05ee7fc640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05d4078e00 0x7f05d407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f06054c0640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05d4078e00 0x7f05d407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f06054c0640 10 monclient: get_auth_request con 0x7f05d4078e00 auth_method 0 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f06054c0640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05d4078e00 0x7f05d407b2c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007b60 tx=0x7f05f0007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f05ee7fc640 10 client.5077.objecter ms_handle_connect 0x7f05d4078e00 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f05ee7fc640 1 -- 10.20.193.6:0/4077131225 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f05fc084070 con 0x7f060015a5e0 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f05ee7fc640 10 client.5077.objecter ms_dispatch2 0x7f0600000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:34.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f05ee7fc640 3 client.5077.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:34.415 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f05ee7fc640 3 client.5077.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:34.415 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.412+0000 7f05ee7fc640 20 client.5077.objecter dump_active .. 0 homeless 2026-02-22T23:28:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.419+0000 7f05ee7fc640 1 -- 10.20.193.6:0/4077131225 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f05fc088030 con 0x7f060015a5e0 2026-02-22T23:28:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.419+0000 7f05ee7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.419+0000 7f05ee7fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:34.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.504+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0600158610 con 0x7f05d4078e00 2026-02-22T23:28:34.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.505+0000 7f05ee7fc640 1 -- 10.20.193.6:0/4077131225 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45281 (secure 0 0 0) 0x7f0600158610 con 0x7f05d4078e00 2026-02-22T23:28:34.508 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:34.509 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-22T23:28:34.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 10 librados: watch_flush enter 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 10 librados: watch_flush exit 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 20 client.5077.objecter shutdown clearing up homeless session... 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 10 client.5077.objecter successfully canceled tick 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05d4078e00 msgr2=0x7f05d407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05d4078e00 0x7f05d407b2c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05f0007b60 tx=0x7f05f0007ab0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 10 monclient: shutdown 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 msgr2=0x7f0600152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600152d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f05fc036000 tx=0x7f05fc004b40 comp rx=0 tx=0).stop 2026-02-22T23:28:34.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 shutdown_connections 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f05d4078e00 0x7f05d407b2c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f060015a5e0 0x7f0600152d10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f06000c99a0 0x7f060006ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 --2- 10.20.193.6:0/4077131225 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f060006ff20 0x7f060014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 >> 10.20.193.6:0/4077131225 conn(0x7f06000cd780 msgr2=0x7f0600158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 shutdown_connections 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 -- 10.20.193.6:0/4077131225 wait complete. 2026-02-22T23:28:34.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.507+0000 7f060774b640 1 librados: shutdown 2026-02-22T23:28:34.517 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-22T23:28:33.807542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437144,"kb_used_data":5592,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512296,"statfs":{"total":1546188226560,"available":1545740591104,"internally_reserved":0,"allocated":5726208,"data_stored":2779510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25502,"internal_metadata":441424994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002278"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:36.952185+0000","last_change":"2026-02-22T23:23:26.541669+0000","last_active":"2026-02-22T23:25:36.952185+0000","last_peered":"2026-02-22T23:25:36.952185+0000","last_clean":"2026-02-22T23:25:36.952185+0000","last_became_active":"2026-02-22T23:23:24.999148+0000","last_became_peered":"2026-02-22T23:23:24.999148+0000","last_unstale":"2026-02-22T23:25:36.952185+0000","last_undegraded":"2026-02-22T23:25:36.952185+0000","last_fullsized":"2026-02-22T23:25:36.952185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:33:54.198723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033743088999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:34.970590+0000","last_change":"2026-02-22T23:23:26.948831+0000","last_active":"2026-02-22T23:25:34.970590+0000","last_peered":"2026-02-22T23:25:34.970590+0000","last_clean":"2026-02-22T23:25:34.970590+0000","last_became_active":"2026-02-22T23:23:24.966693+0000","last_became_peered":"2026-02-22T23:23:24.966693+0000","last_unstale":"2026-02-22T23:25:34.970590+0000","last_undegraded":"2026-02-22T23:25:34.970590+0000","last_fullsized":"2026-02-22T23:25:34.970590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:30:16.311433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033152011000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:30.040017+0000","last_change":"2026-02-22T23:23:26.535885+0000","last_active":"2026-02-22T23:25:30.040017+0000","last_peered":"2026-02-22T23:25:30.040017+0000","last_clean":"2026-02-22T23:25:30.040017+0000","last_became_active":"2026-02-22T23:23:24.806260+0000","last_became_peered":"2026-02-22T23:23:24.806260+0000","last_unstale":"2026-02-22T23:25:30.040017+0000","last_undegraded":"2026-02-22T23:25:30.040017+0000","last_fullsized":"2026-02-22T23:25:30.040017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:46:46.747409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032290068999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:30.011204+0000","last_change":"2026-02-22T23:23:26.547663+0000","last_active":"2026-02-22T23:25:30.011204+0000","last_peered":"2026-02-22T23:25:30.011204+0000","last_clean":"2026-02-22T23:25:30.011204+0000","last_became_active":"2026-02-22T23:23:24.706162+0000","last_became_peered":"2026-02-22T23:23:24.706162+0000","last_unstale":"2026-02-22T23:25:30.011204+0000","last_undegraded":"2026-02-22T23:25:30.011204+0000","last_fullsized":"2026-02-22T23:25:30.011204+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:37:21.100237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041859805999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"27'2","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.979297+0000","last_change":"2026-02-22T23:23:26.386245+0000","last_active":"2026-02-22T23:25:31.979297+0000","last_peered":"2026-02-22T23:25:31.979297+0000","last_clean":"2026-02-22T23:25:31.979297+0000","last_became_active":"2026-02-22T23:23:25.054022+0000","last_became_peered":"2026-02-22T23:23:25.054022+0000","last_unstale":"2026-02-22T23:25:31.979297+0000","last_undegraded":"2026-02-22T23:25:31.979297+0000","last_fullsized":"2026-02-22T23:25:31.979297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:02:19.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.070943361999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.952620+0000","last_change":"2026-02-22T23:23:26.542103+0000","last_active":"2026-02-22T23:25:31.952620+0000","last_peered":"2026-02-22T23:25:31.952620+0000","last_clean":"2026-02-22T23:25:31.952620+0000","last_became_active":"2026-02-22T23:23:24.462347+0000","last_became_peered":"2026-02-22T23:23:24.462347+0000","last_unstale":"2026-02-22T23:25:31.952620+0000","last_undegraded":"2026-02-22T23:25:31.952620+0000","last_fullsized":"2026-02-22T23:25:31.952620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:09:34.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033702211000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:26.977309+0000","last_change":"2026-02-22T23:23:26.382023+0000","last_active":"2026-02-22T23:25:26.977309+0000","last_peered":"2026-02-22T23:25:26.977309+0000","last_clean":"2026-02-22T23:25:26.977309+0000","last_became_active":"2026-02-22T23:23:24.464964+0000","last_became_peered":"2026-02-22T23:23:24.464964+0000","last_unstale":"2026-02-22T23:25:26.977309+0000","last_undegraded":"2026-02-22T23:25:26.977309+0000","last_fullsized":"2026-02-22T23:25:26.977309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:28:59.922786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055238717,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'1","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.988710+0000","last_change":"2026-02-22T23:23:26.538271+0000","last_active":"2026-02-22T23:25:31.988710+0000","last_peered":"2026-02-22T23:25:31.988710+0000","last_clean":"2026-02-22T23:25:31.988710+0000","last_became_active":"2026-02-22T23:23:24.503509+0000","last_became_peered":"2026-02-22T23:23:24.503509+0000","last_unstale":"2026-02-22T23:25:31.988710+0000","last_undegraded":"2026-02-22T23:25:31.988710+0000","last_fullsized":"2026-02-22T23:25:31.988710+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:47:22.484108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032869134000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"27'32","reported_seq":66,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:21.948881+0000","last_change":"2026-02-22T23:23:22.760169+0000","last_active":"2026-02-22T23:25:21.948881+0000","last_peered":"2026-02-22T23:25:21.948881+0000","last_clean":"2026-02-22T23:25:21.948881+0000","last_became_active":"2026-02-22T23:23:22.740754+0000","last_became_peered":"2026-02-22T23:23:22.740754+0000","last_unstale":"2026-02-22T23:25:21.948881+0000","last_undegraded":"2026-02-22T23:25:21.948881+0000","last_fullsized":"2026-02-22T23:25:21.948881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_clean_scrub_stamp":"2026-02-22T23:23:20.759065+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:14:46.099842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,9,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":15,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2,4,5,6,7,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":22,"seq":94489280578,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,3,4,5,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":6,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,4,5,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":21,"seq":90194313281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[2,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":21,"seq":90194313281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[0,4,6,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280577,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,5,6,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[2,4,5,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,5,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":22,"seq":94489280577,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":22,"seq":94489280578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:28:34.519 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:28:34.520 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.587+0000 7f0655cb0640 10 monclient: get_monmap_and_config 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.587+0000 7f0655cb0640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.587+0000 7f0655cb0640 10 monclient: monmap: 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:34.588184+0000 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:34.588184+0000 2026-02-22T23:28:34.588 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 1 Processor -- start 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 1 -- start start 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient: init 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient: _add_conns ranks=[5,3,0,1,2,4] 2026-02-22T23:28:34.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f0650155f20 0x7f06500cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient(hunting): picked mon.noname-c con 0x7f0650155f20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f0650156300 0x7f06500c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient(hunting): picked mon.noname-a con 0x7f0650156300 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f06500c9fa0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f065008fa10 con 0x7f06500c9fa0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f06500cec60 con 0x7f0650156300 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f06500c95d0 con 0x7f0650155f20 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.588+0000 7f0655cb0640 10 monclient(hunting): authenticate will time out at 1389.315838s 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f0650155f20 0x7f06500cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f0650156300 0x7f06500c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f0650156300 0x7f06500c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49842/0 (socket says 10.20.193.6:49842) 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f0650155f20 0x7f06500cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:57048/0 (socket says 10.20.193.6:57048) 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 1 -- 10.20.193.6:0/3219502509 learned_addr learned my addr 10.20.193.6:0/3219502509 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 10 monclient(hunting): get_auth_request con 0x7f0650156300 auth_method 0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 10 monclient(hunting): get_auth_request con 0x7f0650155f20 auth_method 0 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.589+0000 7f064effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 10 monclient(hunting): handle_auth_done global_id 5216 payload 274 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 1 -- 10.20.193.6:0/3219502509 >> v2:10.20.193.6:3300/0 conn(0x7f0650156300 msgr2=0x7f06500c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 1 --2- 10.20.193.6:0/3219502509 >> v2:10.20.193.6:3300/0 conn(0x7f0650156300 0x7f06500c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 1 -- 10.20.193.6:0/3219502509 >> v1:10.20.193.6:6789/0 conn(0x7f06500c9fa0 legacy=0x7f065015bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 1 monclient: found mon.noname-c 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:34.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.590+0000 7f064f7fe640 1 -- 10.20.193.6:0/3219502509 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06500c8ce0 con 0x7f0650155f20 2026-02-22T23:28:34.592 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.591+0000 7f064f7fe640 1 --2- 10.20.193.6:0/3219502509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650155f20 0x7f06500cb6d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f065008f690 tx=0x7f063c02eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb48335bf53f75a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 1 -- 10.20.193.6:0/3219502509 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c03c070 con 0x7f0650155f20 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: dump: 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 5 monclient: authenticate success, global_id 5216 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 1 -- 10.20.193.6:0/3219502509 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c02fae0 con 0x7f0650155f20 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 1 -- 10.20.193.6:0/3219502509 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c037440 con 0x7f0650155f20 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f064e7fc640 10 monclient: dump: 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.595 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 4 set_mon_vals no callback set 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 10 monclient: shutdown 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 -- 10.20.193.6:0/3219502509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650155f20 msgr2=0x7f06500cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3219502509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650155f20 0x7f06500cb6d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f065008f690 tx=0x7f063c02eeb0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 -- 10.20.193.6:0/3219502509 shutdown_connections 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3219502509 >> v2:10.20.193.6:3300/0 conn(0x7f0650156300 0x7f06500c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3219502509 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650155f20 0x7f06500cb6d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 -- 10.20.193.6:0/3219502509 >> 10.20.193.6:0/3219502509 conn(0x7f0650072fc0 msgr2=0x7f06500733d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 -- 10.20.193.6:0/3219502509 shutdown_connections 2026-02-22T23:28:34.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.593+0000 7f0655cb0640 1 -- 10.20.193.6:0/3219502509 wait complete. 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 10 monclient: monmap: 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 librados: starting msgr at 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 librados: starting objecter 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 Processor -- start 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 -- start start 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 librados: setting wanted keys 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 librados: calling monclient init 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 10 monclient: init 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f06500c9fa0 0x7f06501489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 10 monclient(hunting): picked mon.noname-b con 0x7f06500c9fa0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0650155f20 0x7f06501f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.594+0000 7f0655cb0640 10 monclient(hunting): picked mon.noname-a con 0x7f0650155f20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650156300 0x7f0650154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 10 monclient(hunting): picked mon.noname-c con 0x7f0650156300 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f06500c9260 con 0x7f0650155f20 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f06500c8f80 con 0x7f06500c9fa0 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f06500cbc10 con 0x7f0650156300 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f0655cb0640 10 monclient(hunting): authenticate will time out at 1389.322012s 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0650155f20 0x7f06501f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650156300 0x7f0650154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0650155f20 0x7f06501f5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49852/0 (socket says 10.20.193.6:49852) 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650156300 0x7f0650154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:57062/0 (socket says 10.20.193.6:57062) 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 1 -- 10.20.193.6:0/3934419432 learned_addr learned my addr 10.20.193.6:0/3934419432 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 10 monclient(hunting): get_auth_request con 0x7f0650155f20 auth_method 0 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 10 monclient(hunting): get_auth_request con 0x7f0650156300 auth_method 0 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064f7fe640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f06500c9fa0 0x7f06501489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064ffff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064f7fe640 10 monclient(hunting): get_auth_request con 0x7f06500c9fa0 auth_method 0 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064f7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064f7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064f7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.595+0000 7f064effd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064f7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064f7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064f7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 10 monclient(hunting): handle_auth_done global_id 5219 payload 824 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 1 -- 10.20.193.6:0/3934419432 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f06500c9fa0 msgr2=0x7f06501489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f06500c9fa0 0x7f06501489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 1 -- 10.20.193.6:0/3934419432 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0650155f20 msgr2=0x7f06501f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0650155f20 0x7f06501f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 1 monclient: found mon.noname-c 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:34.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.596+0000 7f064ffff640 1 -- 10.20.193.6:0/3934419432 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0650147c60 con 0x7f0650156300 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.597+0000 7f064ffff640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650156300 0x7f0650154b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f064400be20 tx=0x7f064400bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.598+0000 7f062bfff640 10 client.?.objecter ms_handle_connect 0x7f0650156300 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.598+0000 7f062bfff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 1 -- 10.20.193.6:0/3934419432 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644006e60 con 0x7f0650156300 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: dump: 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 1 -- 10.20.193.6:0/3934419432 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0644010070 con 0x7f0650156300 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 5 monclient: authenticate success, global_id 5219 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 10 monclient: _renew_subs 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:34.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06501e8fb0 con 0x7f0650156300 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 1 -- 10.20.193.6:0/3934419432 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064400ccf0 con 0x7f0650156300 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f062bfff640 10 monclient: dump: 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f064dffb640 4 set_mon_vals no callback set 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 10 client.5219.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 10 monclient: _renew_subs 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0650147f00 con 0x7f0650156300 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.599+0000 7f0655cb0640 1 librados: init done 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.600+0000 7f0655cb0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.600+0000 7f0655cb0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.600+0000 7f0655cb0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:34.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.600+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0650077010 con 0x7f0650156300 2026-02-22T23:28:34.700 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.699+0000 7f062bfff640 1 -- 10.20.193.6:0/3934419432 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0644004070 con 0x7f0650156300 2026-02-22T23:28:34.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f062bfff640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0620078e50 0x7f062007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f064f7fe640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0620078e50 0x7f062007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f064f7fe640 10 monclient: get_auth_request con 0x7f0620078e50 auth_method 0 2026-02-22T23:28:34.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f062bfff640 1 -- 10.20.193.6:0/3934419432 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f064405dbc0 con 0x7f0650156300 2026-02-22T23:28:34.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f062bfff640 10 client.5219.objecter ms_dispatch2 0x7f0650000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:34.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f062bfff640 3 client.5219.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:34.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f062bfff640 3 client.5219.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:34.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.700+0000 7f062bfff640 20 client.5219.objecter dump_active .. 0 homeless 2026-02-22T23:28:34.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.701+0000 7f064f7fe640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0620078e50 0x7f062007b310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f063c039000 tx=0x7f063c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.703 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.701+0000 7f062bfff640 10 client.5219.objecter ms_handle_connect 0x7f0620078e50 2026-02-22T23:28:34.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.706+0000 7f062bfff640 1 -- 10.20.193.6:0/3934419432 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f0650077010 con 0x7f0650156300 2026-02-22T23:28:34.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.707+0000 7f062bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.707+0000 7f062bfff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:34.792 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.791+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06500cf170 con 0x7f0620078e50 2026-02-22T23:28:34.794 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.793+0000 7f062bfff640 1 -- 10.20.193.6:0/3934419432 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45281 (secure 0 0 0) 0x7f06500cf170 con 0x7f0620078e50 2026-02-22T23:28:34.794 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:34.796 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-22T23:28:34.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 10 librados: watch_flush enter 2026-02-22T23:28:34.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 10 librados: watch_flush exit 2026-02-22T23:28:34.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 20 client.5219.objecter shutdown clearing up homeless session... 2026-02-22T23:28:34.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 10 client.5219.objecter successfully canceled tick 2026-02-22T23:28:34.796 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0620078e50 msgr2=0x7f062007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0620078e50 0x7f062007b310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f063c039000 tx=0x7f063c033000 comp rx=0 tx=0).stop 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 10 monclient: shutdown 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650156300 msgr2=0x7f0650154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650156300 0x7f0650154b30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f064400be20 tx=0x7f064400bf20 comp rx=0 tx=0).stop 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 shutdown_connections 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0620078e50 0x7f062007b310 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0650156300 0x7f0650154b30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0650155f20 0x7f06501f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 --2- 10.20.193.6:0/3934419432 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f06500c9fa0 0x7f06501489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 >> 10.20.193.6:0/3934419432 conn(0x7f0650072fc0 msgr2=0x7f0650152370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:34.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 shutdown_connections 2026-02-22T23:28:34.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 -- 10.20.193.6:0/3934419432 wait complete. 2026-02-22T23:28:34.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.794+0000 7f0655cb0640 1 librados: shutdown 2026-02-22T23:28:34.803 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2026-02-22T23:28:33.807542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437144,"kb_used_data":5592,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512296,"statfs":{"total":1546188226560,"available":1545740591104,"internally_reserved":0,"allocated":5726208,"data_stored":2779510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25502,"internal_metadata":441424994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002278"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:36.952185+0000","last_change":"2026-02-22T23:23:26.541669+0000","last_active":"2026-02-22T23:25:36.952185+0000","last_peered":"2026-02-22T23:25:36.952185+0000","last_clean":"2026-02-22T23:25:36.952185+0000","last_became_active":"2026-02-22T23:23:24.999148+0000","last_became_peered":"2026-02-22T23:23:24.999148+0000","last_unstale":"2026-02-22T23:25:36.952185+0000","last_undegraded":"2026-02-22T23:25:36.952185+0000","last_fullsized":"2026-02-22T23:25:36.952185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:33:54.198723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033743088999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:34.970590+0000","last_change":"2026-02-22T23:23:26.948831+0000","last_active":"2026-02-22T23:25:34.970590+0000","last_peered":"2026-02-22T23:25:34.970590+0000","last_clean":"2026-02-22T23:25:34.970590+0000","last_became_active":"2026-02-22T23:23:24.966693+0000","last_became_peered":"2026-02-22T23:23:24.966693+0000","last_unstale":"2026-02-22T23:25:34.970590+0000","last_undegraded":"2026-02-22T23:25:34.970590+0000","last_fullsized":"2026-02-22T23:25:34.970590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:30:16.311433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033152011000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:30.040017+0000","last_change":"2026-02-22T23:23:26.535885+0000","last_active":"2026-02-22T23:25:30.040017+0000","last_peered":"2026-02-22T23:25:30.040017+0000","last_clean":"2026-02-22T23:25:30.040017+0000","last_became_active":"2026-02-22T23:23:24.806260+0000","last_became_peered":"2026-02-22T23:23:24.806260+0000","last_unstale":"2026-02-22T23:25:30.040017+0000","last_undegraded":"2026-02-22T23:25:30.040017+0000","last_fullsized":"2026-02-22T23:25:30.040017+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T04:46:46.747409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032290068999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:30.011204+0000","last_change":"2026-02-22T23:23:26.547663+0000","last_active":"2026-02-22T23:25:30.011204+0000","last_peered":"2026-02-22T23:25:30.011204+0000","last_clean":"2026-02-22T23:25:30.011204+0000","last_became_active":"2026-02-22T23:23:24.706162+0000","last_became_peered":"2026-02-22T23:23:24.706162+0000","last_unstale":"2026-02-22T23:25:30.011204+0000","last_undegraded":"2026-02-22T23:25:30.011204+0000","last_fullsized":"2026-02-22T23:25:30.011204+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:37:21.100237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041859805999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"27'2","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.979297+0000","last_change":"2026-02-22T23:23:26.386245+0000","last_active":"2026-02-22T23:25:31.979297+0000","last_peered":"2026-02-22T23:25:31.979297+0000","last_clean":"2026-02-22T23:25:31.979297+0000","last_became_active":"2026-02-22T23:23:25.054022+0000","last_became_peered":"2026-02-22T23:23:25.054022+0000","last_unstale":"2026-02-22T23:25:31.979297+0000","last_undegraded":"2026-02-22T23:25:31.979297+0000","last_fullsized":"2026-02-22T23:25:31.979297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:02:19.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.070943361999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.952620+0000","last_change":"2026-02-22T23:23:26.542103+0000","last_active":"2026-02-22T23:25:31.952620+0000","last_peered":"2026-02-22T23:25:31.952620+0000","last_clean":"2026-02-22T23:25:31.952620+0000","last_became_active":"2026-02-22T23:23:24.462347+0000","last_became_peered":"2026-02-22T23:23:24.462347+0000","last_unstale":"2026-02-22T23:25:31.952620+0000","last_undegraded":"2026-02-22T23:25:31.952620+0000","last_fullsized":"2026-02-22T23:25:31.952620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:09:34.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033702211000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:26.977309+0000","last_change":"2026-02-22T23:23:26.382023+0000","last_active":"2026-02-22T23:25:26.977309+0000","last_peered":"2026-02-22T23:25:26.977309+0000","last_clean":"2026-02-22T23:25:26.977309+0000","last_became_active":"2026-02-22T23:23:24.464964+0000","last_became_peered":"2026-02-22T23:23:24.464964+0000","last_unstale":"2026-02-22T23:25:26.977309+0000","last_undegraded":"2026-02-22T23:25:26.977309+0000","last_fullsized":"2026-02-22T23:25:26.977309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:28:59.922786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055238717,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'1","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.988710+0000","last_change":"2026-02-22T23:23:26.538271+0000","last_active":"2026-02-22T23:25:31.988710+0000","last_peered":"2026-02-22T23:25:31.988710+0000","last_clean":"2026-02-22T23:25:31.988710+0000","last_became_active":"2026-02-22T23:23:24.503509+0000","last_became_peered":"2026-02-22T23:23:24.503509+0000","last_unstale":"2026-02-22T23:25:31.988710+0000","last_undegraded":"2026-02-22T23:25:31.988710+0000","last_fullsized":"2026-02-22T23:25:31.988710+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T23:47:22.484108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032869134000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"27'32","reported_seq":66,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:21.948881+0000","last_change":"2026-02-22T23:23:22.760169+0000","last_active":"2026-02-22T23:25:21.948881+0000","last_peered":"2026-02-22T23:25:21.948881+0000","last_clean":"2026-02-22T23:25:21.948881+0000","last_became_active":"2026-02-22T23:23:22.740754+0000","last_became_peered":"2026-02-22T23:23:22.740754+0000","last_unstale":"2026-02-22T23:25:21.948881+0000","last_undegraded":"2026-02-22T23:25:21.948881+0000","last_fullsized":"2026-02-22T23:25:21.948881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_clean_scrub_stamp":"2026-02-22T23:23:20.759065+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:14:46.099842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":23,"seq":98784247874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,9,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":15,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2,4,5,6,7,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":22,"seq":94489280578,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,3,4,5,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":6,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,4,5,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":21,"seq":90194313281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[2,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313280,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":21,"seq":90194313281,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[0,4,6,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":21,"seq":90194313281,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280577,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,5,6,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[2,4,5,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,5,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":22,"seq":94489280577,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":22,"seq":94489280578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:28:34.805 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:28:34.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.872+0000 7fdf702b0640 10 monclient: get_monmap_and_config 2026-02-22T23:28:34.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.872+0000 7fdf702b0640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.872+0000 7fdf702b0640 10 monclient: monmap: 2026-02-22T23:28:34.874 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.874 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.874 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:34.873514+0000 2026-02-22T23:28:34.874 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:34.873514+0000 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:34.875 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 1 Processor -- start 2026-02-22T23:28:34.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 1 -- start start 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient: init 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient: _add_conns ranks=[5,3,2,4,0,1] 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fdf6806ff20 0x7fdf68070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient(hunting): picked mon.noname-c con 0x7fdf6806ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fdf680c99a0 0x7fdf68159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient(hunting): picked mon.noname-a con 0x7fdf680c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fdf6815a410 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf6815a860 con 0x7fdf6815a410 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fdf68159a40 con 0x7fdf680c99a0 2026-02-22T23:28:34.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdf680c7e90 con 0x7fdf6806ff20 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.873+0000 7fdf702b0640 10 monclient(hunting): authenticate will time out at 1389.600876s 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6e025640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fdf6806ff20 0x7fdf68070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fdf680c99a0 0x7fdf68159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fdf680c99a0 0x7fdf68159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49864/0 (socket says 10.20.193.6:49864) 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 1 -- 10.20.193.6:0/3173302012 learned_addr learned my addr 10.20.193.6:0/3173302012 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 10 monclient(hunting): get_auth_request con 0x7fdf680c99a0 auth_method 0 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6e025640 10 monclient(hunting): get_auth_request con 0x7fdf6806ff20 auth_method 0 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6e025640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6e025640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6e025640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.874+0000 7fdf6d824640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6e025640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6e025640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6e025640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 10 monclient(hunting): handle_auth_done global_id 5133 payload 274 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 1 -- 10.20.193.6:0/3173302012 >> v2:10.20.193.80:3300/0 conn(0x7fdf6806ff20 msgr2=0x7fdf68070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 1 --2- 10.20.193.6:0/3173302012 >> v2:10.20.193.80:3300/0 conn(0x7fdf6806ff20 0x7fdf68070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 1 -- 10.20.193.6:0/3173302012 >> v1:10.20.193.80:6789/0 conn(0x7fdf6815a410 legacy=0x7fdf6814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 1 monclient: found mon.noname-a 2026-02-22T23:28:34.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:34.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.875+0000 7fdf6d824640 1 -- 10.20.193.6:0/3173302012 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf680c8170 con 0x7fdf680c99a0 2026-02-22T23:28:34.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.876+0000 7fdf6d824640 1 --2- 10.20.193.6:0/3173302012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf68159ed0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009970 tx=0x7fdf5c02eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddab0ba29a6f619 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 1 -- 10.20.193.6:0/3173302012 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c03c070 con 0x7fdf680c99a0 2026-02-22T23:28:34.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:34.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: dump: 2026-02-22T23:28:34.888 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.888 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.888 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 1 -- 10.20.193.6:0/3173302012 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf5c02fbc0 con 0x7fdf680c99a0 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 1 -- 10.20.193.6:0/3173302012 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c037440 con 0x7fdf680c99a0 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf6d023640 10 monclient: dump: 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.890 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf702b0640 5 monclient: authenticate success, global_id 5133 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf702b0640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf702b0640 4 set_mon_vals no callback set 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf702b0640 10 monclient: shutdown 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf702b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf702b0640 1 -- 10.20.193.6:0/3173302012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 msgr2=0x7fdf68159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.878+0000 7fdf702b0640 1 --2- 10.20.193.6:0/3173302012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf68159ed0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009970 tx=0x7fdf5c02eef0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 -- 10.20.193.6:0/3173302012 shutdown_connections 2026-02-22T23:28:34.891 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 --2- 10.20.193.6:0/3173302012 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf68159ed0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 --2- 10.20.193.6:0/3173302012 >> v2:10.20.193.80:3300/0 conn(0x7fdf6806ff20 0x7fdf68070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 -- 10.20.193.6:0/3173302012 >> 10.20.193.6:0/3173302012 conn(0x7fdf680cd780 msgr2=0x7fdf680cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 -- 10.20.193.6:0/3173302012 shutdown_connections 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 -- 10.20.193.6:0/3173302012 wait complete. 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 10 monclient: build_initial_monmap 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 10 monclient: monmap: 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.892 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 librados: starting msgr at 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 librados: starting objecter 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 Processor -- start 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 -- start start 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 librados: setting wanted keys 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 1 librados: calling monclient init 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.879+0000 7fdf702b0640 10 monclient: init 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:34.893 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fdf6806ff20 0x7fdf681f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): picked mon.noname-c con 0x7fdf6806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf681f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): picked mon.noname-a con 0x7fdf680c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fdf6815a410 0x7fdf681f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): picked mon.noname-b con 0x7fdf6815a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680c8a10 con 0x7fdf680c99a0 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680c8730 con 0x7fdf6815a410 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68150520 con 0x7fdf6806ff20 2026-02-22T23:28:34.894 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf702b0640 10 monclient(hunting): authenticate will time out at 1389.607363s 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6d824640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf681f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6d824640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf681f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49878/0 (socket says 10.20.193.6:49878) 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6d824640 1 -- 10.20.193.6:0/1333442794 learned_addr learned my addr 10.20.193.6:0/1333442794 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6d824640 10 monclient(hunting): get_auth_request con 0x7fdf680c99a0 auth_method 0 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6d824640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6d824640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6d824640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6e826640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fdf6815a410 0x7fdf681f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.880+0000 7fdf6e025640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fdf6806ff20 0x7fdf681f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6d824640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6d824640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6d824640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e025640 10 monclient(hunting): get_auth_request con 0x7fdf6806ff20 auth_method 0 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e025640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e025640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e025640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e826640 10 monclient(hunting): get_auth_request con 0x7fdf6815a410 auth_method 0 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e826640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e826640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e826640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e025640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e025640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e025640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.896 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e826640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e826640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.881+0000 7fdf6e826640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 10 monclient(hunting): handle_auth_done global_id 5136 payload 824 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 1 -- 10.20.193.6:0/1333442794 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fdf6806ff20 msgr2=0x7fdf681f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fdf6806ff20 0x7fdf681f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 1 -- 10.20.193.6:0/1333442794 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fdf6815a410 msgr2=0x7fdf681f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fdf6815a410 0x7fdf681f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 10 monclient: _finish_hunting 0 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 1 monclient: found mon.noname-a 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:34.897 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.882+0000 7fdf6d824640 1 -- 10.20.193.6:0/1333442794 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf681f51d0 con 0x7fdf680c99a0 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.883+0000 7fdf6d824640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf681f4b70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c02f4a0 tx=0x7fdf5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.883+0000 7fdf4effd640 10 client.?.objecter ms_handle_connect 0x7fdf680c99a0 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.883+0000 7fdf4effd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 1 -- 10.20.193.6:0/1333442794 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c03c040 con 0x7fdf680c99a0 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: dump: 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.898 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: _finish_auth 0 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 1 -- 10.20.193.6:0/1333442794 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf5c042410 con 0x7fdf680c99a0 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 1 -- 10.20.193.6:0/1333442794 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0427e0 con 0x7fdf680c99a0 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:34.899 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf4effd640 10 monclient: dump: 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf6c822640 4 set_mon_vals no callback set 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 5 monclient: authenticate success, global_id 5136 2026-02-22T23:28:34.900 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 10 monclient: _renew_subs 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf681f54b0 con 0x7fdf680c99a0 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 10 client.5136.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 10 monclient: _renew_subs 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681f5a70 con 0x7fdf680c99a0 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.884+0000 7fdf702b0640 1 librados: init done 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.885+0000 7fdf702b0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.885+0000 7fdf702b0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.885+0000 7fdf702b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:34.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.885+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf68070300 con 0x7fdf680c99a0 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.896+0000 7fdf4effd640 1 -- 10.20.193.6:0/1333442794 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fdf5c042980 con 0x7fdf680c99a0 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.896+0000 7fdf4effd640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fdf38078e00 0x7fdf3807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf6e025640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fdf38078e00 0x7fdf3807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf6e025640 10 monclient: get_auth_request con 0x7fdf38078e00 auth_method 0 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf6e025640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fdf38078e00 0x7fdf3807b2c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009c40 tx=0x7fdf58009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf4effd640 10 client.5136.objecter ms_handle_connect 0x7fdf38078e00 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf4effd640 1 -- 10.20.193.6:0/1333442794 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fdf5c083650 con 0x7fdf680c99a0 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf4effd640 10 client.5136.objecter ms_dispatch2 0x7fdf68000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf4effd640 3 client.5136.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:34.903 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf4effd640 3 client.5136.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:34.904 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.897+0000 7fdf4effd640 20 client.5136.objecter dump_active .. 0 homeless 2026-02-22T23:28:34.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.994+0000 7fdf4effd640 1 -- 10.20.193.6:0/1333442794 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fdf5c087610 con 0x7fdf680c99a0 2026-02-22T23:28:34.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.994+0000 7fdf4effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:34.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:34.994+0000 7fdf4effd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:35.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.078+0000 7fdf702b0640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:28:35.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.078+0000 7fdf702b0640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:28:35.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.078+0000 7fdf702b0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:35.080 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.078+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf6814feb0 con 0x7fdf680c99a0 2026-02-22T23:28:35.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.096+0000 7fdf4effd640 1 -- 10.20.193.6:0/1333442794 <== mon.0 v2:10.20.193.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+24580 (secure 0 0 0) 0x7fdf5c08b910 con 0x7fdf680c99a0 2026-02-22T23:28:35.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.096+0000 7fdf4effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2026-02-22T23:28:35.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.096+0000 7fdf4effd640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:28:35.098 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:35.098 INFO:teuthology.orchestra.run.trial006.stdout:{"epoch":28,"fsid":"f24f9547-07c5-4c58-99d4-b3bf9b186dd1","created":"2026-02-22T23:21:38.943861+0000","modified":"2026-02-22T23:23:32.863335+0000","last_up_change":"2026-02-22T23:23:21.874564+0000","last_in_change":"2026-02-22T23:21:45.101496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:23:19.944772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T23:23:22.821753+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0100002288818359,"score_stable":4.0100002288818359,"optimal_score":0.81000000238418579,"raw_score_acting":3.25,"raw_score_stable":3.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"02ed71aa-ff9a-4f06-b3fc-8fe1ec77ae64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6824","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6825","nonce":1913492598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6826","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6827","nonce":1913492598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6830","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6831","nonce":1913492598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6828","nonce":1913492598},{"type":"v1","addr":"10.20.193.6:6829","nonce":1913492598}]},"public_addr":"10.20.193.6:6825/1913492598","cluster_addr":"10.20.193.6:6827/1913492598","heartbeat_back_addr":"10.20.193.6:6831/1913492598","heartbeat_front_addr":"10.20.193.6:6829/1913492598","state":["exists","up"]},{"osd":1,"uuid":"d0777c8c-8788-440c-b557-b72b5f0ec2f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6824","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6825","nonce":2813004313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6826","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6827","nonce":2813004313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6830","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6831","nonce":2813004313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6828","nonce":2813004313},{"type":"v1","addr":"10.20.193.40:6829","nonce":2813004313}]},"public_addr":"10.20.193.40:6825/2813004313","cluster_addr":"10.20.193.40:6827/2813004313","heartbeat_back_addr":"10.20.193.40:6831/2813004313","heartbeat_front_addr":"10.20.193.40:6829/2813004313","state":["exists","up"]},{"osd":2,"uuid":"b0c8c31e-3748-43a4-ae36-aa42fd9a5030","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6817","nonce":1223585399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6818","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6819","nonce":1223585399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6822","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6823","nonce":1223585399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":1223585399},{"type":"v1","addr":"10.20.193.80:6821","nonce":1223585399}]},"public_addr":"10.20.193.80:6817/1223585399","cluster_addr":"10.20.193.80:6819/1223585399","heartbeat_back_addr":"10.20.193.80:6823/1223585399","heartbeat_front_addr":"10.20.193.80:6821/1223585399","state":["exists","up"]},{"osd":3,"uuid":"542e6008-d73b-45c7-a6d1-f34712ec1541","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6818","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6819","nonce":2996878330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6820","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6821","nonce":2996878330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6824","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6825","nonce":2996878330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6822","nonce":2996878330},{"type":"v1","addr":"10.20.193.189:6823","nonce":2996878330}]},"public_addr":"10.20.193.189:6819/2996878330","cluster_addr":"10.20.193.189:6821/2996878330","heartbeat_back_addr":"10.20.193.189:6825/2996878330","heartbeat_front_addr":"10.20.193.189:6823/2996878330","state":["exists","up"]},{"osd":4,"uuid":"075bb353-79d1-4bb1-b765-5b243bde4a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6800","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6801","nonce":2180113666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6802","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6803","nonce":2180113666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6806","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6807","nonce":2180113666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6804","nonce":2180113666},{"type":"v1","addr":"10.20.193.6:6805","nonce":2180113666}]},"public_addr":"10.20.193.6:6801/2180113666","cluster_addr":"10.20.193.6:6803/2180113666","heartbeat_back_addr":"10.20.193.6:6807/2180113666","heartbeat_front_addr":"10.20.193.6:6805/2180113666","state":["exists","up"]},{"osd":5,"uuid":"52c5895d-8795-4e32-8b07-c4f246dc0503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6800","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6801","nonce":2956321569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6802","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6803","nonce":2956321569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6806","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6807","nonce":2956321569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6804","nonce":2956321569},{"type":"v1","addr":"10.20.193.40:6805","nonce":2956321569}]},"public_addr":"10.20.193.40:6801/2956321569","cluster_addr":"10.20.193.40:6803/2956321569","heartbeat_back_addr":"10.20.193.40:6807/2956321569","heartbeat_front_addr":"10.20.193.40:6805/2956321569","state":["exists","up"]},{"osd":6,"uuid":"6f2925ef-1371-4d00-8ac7-9efd4ef71fe6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6801","nonce":144861518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6803","nonce":144861518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6807","nonce":144861518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":144861518},{"type":"v1","addr":"10.20.193.80:6805","nonce":144861518}]},"public_addr":"10.20.193.80:6801/144861518","cluster_addr":"10.20.193.80:6803/144861518","heartbeat_back_addr":"10.20.193.80:6807/144861518","heartbeat_front_addr":"10.20.193.80:6805/144861518","state":["exists","up"]},{"osd":7,"uuid":"963d277c-a350-4d8b-b94f-83365e8c8a9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6810","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6811","nonce":980553521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6812","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6813","nonce":980553521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6816","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6817","nonce":980553521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6814","nonce":980553521},{"type":"v1","addr":"10.20.193.189:6815","nonce":980553521}]},"public_addr":"10.20.193.189:6811/980553521","cluster_addr":"10.20.193.189:6813/980553521","heartbeat_back_addr":"10.20.193.189:6817/980553521","heartbeat_front_addr":"10.20.193.189:6815/980553521","state":["exists","up"]},{"osd":8,"uuid":"8ccd8ee4-5de1-4fb7-a18a-c1b8cd1b8b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6816","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6817","nonce":1002652796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6818","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6819","nonce":1002652796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6822","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6823","nonce":1002652796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6820","nonce":1002652796},{"type":"v1","addr":"10.20.193.6:6821","nonce":1002652796}]},"public_addr":"10.20.193.6:6817/1002652796","cluster_addr":"10.20.193.6:6819/1002652796","heartbeat_back_addr":"10.20.193.6:6823/1002652796","heartbeat_front_addr":"10.20.193.6:6821/1002652796","state":["exists","up"]},{"osd":9,"uuid":"528c9e90-8c81-4193-a32a-6809648cf450","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6808","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6809","nonce":979607499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6810","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6811","nonce":979607499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6814","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6815","nonce":979607499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6812","nonce":979607499},{"type":"v1","addr":"10.20.193.40:6813","nonce":979607499}]},"public_addr":"10.20.193.40:6809/979607499","cluster_addr":"10.20.193.40:6811/979607499","heartbeat_back_addr":"10.20.193.40:6815/979607499","heartbeat_front_addr":"10.20.193.40:6813/979607499","state":["exists","up"]},{"osd":10,"uuid":"acd57453-2064-44a4-bb36-a8c2d2c13c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6809","nonce":2974128537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6810","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6811","nonce":2974128537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6814","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6815","nonce":2974128537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2974128537},{"type":"v1","addr":"10.20.193.80:6813","nonce":2974128537}]},"public_addr":"10.20.193.80:6809/2974128537","cluster_addr":"10.20.193.80:6811/2974128537","heartbeat_back_addr":"10.20.193.80:6815/2974128537","heartbeat_front_addr":"10.20.193.80:6813/2974128537","state":["exists","up"]},{"osd":11,"uuid":"ea741f02-ae0b-419b-8892-34f4b28f6ab6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6802","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6803","nonce":2070530833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6804","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6805","nonce":2070530833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6808","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6809","nonce":2070530833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6806","nonce":2070530833},{"type":"v1","addr":"10.20.193.189:6807","nonce":2070530833}]},"public_addr":"10.20.193.189:6803/2070530833","cluster_addr":"10.20.193.189:6805/2070530833","heartbeat_back_addr":"10.20.193.189:6809/2070530833","heartbeat_front_addr":"10.20.193.189:6807/2070530833","state":["exists","up"]},{"osd":12,"uuid":"6f306679-2d08-48ea-9fd7-3c4cf4c9323c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6808","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6809","nonce":138679770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6810","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6811","nonce":138679770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6814","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6815","nonce":138679770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.6:6812","nonce":138679770},{"type":"v1","addr":"10.20.193.6:6813","nonce":138679770}]},"public_addr":"10.20.193.6:6809/138679770","cluster_addr":"10.20.193.6:6811/138679770","heartbeat_back_addr":"10.20.193.6:6815/138679770","heartbeat_front_addr":"10.20.193.6:6813/138679770","state":["exists","up"]},{"osd":13,"uuid":"6b0643dc-1483-4635-a16d-adf7e477c900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6816","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6817","nonce":1944897191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6818","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6819","nonce":1944897191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6822","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6823","nonce":1944897191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.40:6820","nonce":1944897191},{"type":"v1","addr":"10.20.193.40:6821","nonce":1944897191}]},"public_addr":"10.20.193.40:6817/1944897191","cluster_addr":"10.20.193.40:6819/1944897191","heartbeat_back_addr":"10.20.193.40:6823/1944897191","heartbeat_front_addr":"10.20.193.40:6821/1944897191","state":["exists","up"]},{"osd":14,"uuid":"cce82a9e-9fa7-4718-ad88-757e1e46acde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6824","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6825","nonce":3509265050}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6826","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6827","nonce":3509265050}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6830","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6831","nonce":3509265050}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6828","nonce":3509265050},{"type":"v1","addr":"10.20.193.80:6829","nonce":3509265050}]},"public_addr":"10.20.193.80:6825/3509265050","cluster_addr":"10.20.193.80:6827/3509265050","heartbeat_back_addr":"10.20.193.80:6831/3509265050","heartbeat_front_addr":"10.20.193.80:6829/3509265050","state":["exists","up"]},{"osd":15,"uuid":"aec0b3b0-8781-48f2-9d8d-a779e70b174f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6826","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6827","nonce":3776650816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6828","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6829","nonce":3776650816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6832","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6833","nonce":3776650816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.189:6830","nonce":3776650816},{"type":"v1","addr":"10.20.193.189:6831","nonce":3776650816}]},"public_addr":"10.20.193.189:6827/3776650816","cluster_addr":"10.20.193.189:6829/3776650816","heartbeat_back_addr":"10.20.193.189:6833/3776650816","heartbeat_front_addr":"10.20.193.189:6831/3776650816","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.797315+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.477029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.595886+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.915624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.060069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.938930+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.614574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.167676+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.905706+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.437161+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.930818+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.151987+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:19.520133+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.819151+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:17.612032+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:23:18.945924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:28:35.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 10 librados: watch_flush enter 2026-02-22T23:28:35.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 10 librados: watch_flush exit 2026-02-22T23:28:35.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 20 client.5136.objecter shutdown clearing up homeless session... 2026-02-22T23:28:35.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 10 client.5136.objecter successfully canceled tick 2026-02-22T23:28:35.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fdf38078e00 msgr2=0x7fdf3807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fdf38078e00 0x7fdf3807b2c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009c40 tx=0x7fdf58009290 comp rx=0 tx=0).stop 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 10 monclient: shutdown 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 msgr2=0x7fdf681f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf681f4b70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c02f4a0 tx=0x7fdf5c0047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 shutdown_connections 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fdf38078e00 0x7fdf3807b2c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fdf6815a410 0x7fdf681f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fdf680c99a0 0x7fdf681f4b70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 --2- 10.20.193.6:0/1333442794 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fdf6806ff20 0x7fdf681f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 >> 10.20.193.6:0/1333442794 conn(0x7fdf680cd780 msgr2=0x7fdf680cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:35.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 shutdown_connections 2026-02-22T23:28:35.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 -- 10.20.193.6:0/1333442794 wait complete. 2026-02-22T23:28:35.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:35.097+0000 7fdf702b0640 1 librados: shutdown 2026-02-22T23:28:36.109 INFO:tasks.ceph:Scrubbing osd.0 2026-02-22T23:28:36.109 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.181+0000 7f837ad2c640 10 monclient: get_monmap_and_config 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.181+0000 7f837ad2c640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.181+0000 7f837ad2c640 10 monclient: monmap: 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:36.181925+0000 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:36.181925+0000 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.182 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.181+0000 7f837ad2c640 1 Processor -- start 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.181+0000 7f837ad2c640 1 -- start start 2026-02-22T23:28:36.183 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.181+0000 7f837ad2c640 10 monclient: init 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient: _add_conns ranks=[3,0,1,2,4,5] 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f83741503e0 0x7f8374159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient(hunting): picked mon.noname-a con 0x7f83741503e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f837415a070 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f83740704c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f837415a570 con 0x7f837415a070 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f837415a850 con 0x7f83740704c0 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f83740c7d70 con 0x7f83741503e0 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f837ad2c640 10 monclient(hunting): authenticate will time out at 1390.909339s 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f83792a2640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f83740704c0 0x7f83740c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:47888/0 (socket says 10.20.193.6:47888) 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f83792a2640 1 -- 10.20.193.6:0/1344306749 learned_addr learned my addr 10.20.193.6:0/1344306749 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.182+0000 7f8378aa1640 1 --2- 10.20.193.6:0/1344306749 >> v2:10.20.193.6:3300/0 conn(0x7f83741503e0 0x7f8374159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.183+0000 7f8378aa1640 10 monclient(hunting): get_auth_request con 0x7f83741503e0 auth_method 0 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.183+0000 7f8378aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.183+0000 7f8378aa1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.183+0000 7f8378aa1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.183+0000 7f8378aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.183+0000 7f8378aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.184 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.183+0000 7f8378aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f836b7fe640 1 -- 10.20.193.6:0/1344306749 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3777990134 0 0) 0x7f837415a850 con 0x7f83740704c0 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f836b7fe640 10 monclient(hunting): _negotiate 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f836b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f836b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f836b7fe640 10 monclient(hunting): my global_id is 5086 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f836b7fe640 1 -- 10.20.193.6:0/1344306749 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8360003620 con 0x7f83740704c0 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 10 monclient(hunting): handle_auth_done global_id 5139 payload 274 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 1 -- 10.20.193.6:0/1344306749 >> v1:10.20.193.40:6789/0 conn(0x7f83740704c0 legacy=0x7f83740c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 1 -- 10.20.193.6:0/1344306749 >> v1:10.20.193.6:6789/0 conn(0x7f837415a070 legacy=0x7f837406fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 1 monclient: found mon.noname-a 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.185 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.184+0000 7f8378aa1640 1 -- 10.20.193.6:0/1344306749 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83740c7fc0 con 0x7f83741503e0 2026-02-22T23:28:36.186 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.185+0000 7f8378aa1640 1 --2- 10.20.193.6:0/1344306749 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f8374159b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f83640098a0 tx=0x7f836402eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc89f80459d3cd9 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 1 -- 10.20.193.6:0/1344306749 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836403c070 con 0x7f83741503e0 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: dump: 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 1 -- 10.20.193.6:0/1344306749 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836402fb70 con 0x7f83741503e0 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 1 -- 10.20.193.6:0/1344306749 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364037440 con 0x7f83741503e0 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 5 monclient: authenticate success, global_id 5139 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 4 set_mon_vals no callback set 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f836b7fe640 10 monclient: dump: 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.189 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 10 monclient: shutdown 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 1 -- 10.20.193.6:0/1344306749 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 msgr2=0x7f8374159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 1 --2- 10.20.193.6:0/1344306749 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f8374159b30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f83640098a0 tx=0x7f836402eeb0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 1 -- 10.20.193.6:0/1344306749 shutdown_connections 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 1 --2- 10.20.193.6:0/1344306749 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f8374159b30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 1 -- 10.20.193.6:0/1344306749 >> 10.20.193.6:0/1344306749 conn(0x7f83740cd600 msgr2=0x7f83740cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 1 -- 10.20.193.6:0/1344306749 shutdown_connections 2026-02-22T23:28:36.190 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.187+0000 7f837ad2c640 1 -- 10.20.193.6:0/1344306749 wait complete. 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 10 monclient: monmap: 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 1 librados: starting msgr at 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 1 librados: starting objecter 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 1 Processor -- start 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 1 -- start start 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 1 librados: setting wanted keys 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 1 librados: calling monclient init 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.188+0000 7f837ad2c640 10 monclient: init 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.191 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f83740704c0 0x7f83741f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): picked mon.noname-b con 0x7f83740704c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f837406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): picked mon.noname-a con 0x7f83741503e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f837415a070 0x7f837406f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): picked mon.noname-c con 0x7f837415a070 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f83741507c0 con 0x7f83741503e0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f83740c8610 con 0x7f83740704c0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8374150940 con 0x7f837415a070 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f837ad2c640 10 monclient(hunting): authenticate will time out at 1390.916123s 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f837406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f837406eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49896/0 (socket says 10.20.193.6:49896) 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 1 -- 10.20.193.6:0/4166733963 learned_addr learned my addr 10.20.193.6:0/4166733963 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 10 monclient(hunting): get_auth_request con 0x7f83741503e0 auth_method 0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f8378aa1640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f83740704c0 0x7f83741f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f83792a2640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f837415a070 0x7f837406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f836bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f8378aa1640 10 monclient(hunting): get_auth_request con 0x7f83740704c0 auth_method 0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f8378aa1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f8378aa1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f8378aa1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f83792a2640 10 monclient(hunting): get_auth_request con 0x7f837415a070 auth_method 0 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f83792a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f83792a2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.189+0000 7f83792a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.192 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.190+0000 7f8378aa1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.190+0000 7f8378aa1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.190+0000 7f8378aa1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.190+0000 7f83792a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.190+0000 7f83792a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.190+0000 7f83792a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 10 monclient(hunting): handle_auth_done global_id 5142 payload 824 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 1 -- 10.20.193.6:0/4166733963 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f837415a070 msgr2=0x7f837406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f837415a070 0x7f837406f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 1 -- 10.20.193.6:0/4166733963 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f83740704c0 msgr2=0x7f83741f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f83740704c0 0x7f83741f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 1 monclient: found mon.noname-a 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.191+0000 7f836bfff640 1 -- 10.20.193.6:0/4166733963 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83741f3e50 con 0x7f83741503e0 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.192+0000 7f836bfff640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f837406eb10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f835c00b4d0 tx=0x7f835c00b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.192+0000 7f83697fa640 10 client.?.objecter ms_handle_connect 0x7f83741503e0 2026-02-22T23:28:36.193 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.192+0000 7f83697fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c004480 con 0x7f83741503e0 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: dump: 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c015070 con 0x7f83741503e0 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c002920 con 0x7f83741503e0 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f83697fa640 10 monclient: dump: 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.195 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f836affd640 4 set_mon_vals no callback set 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 5 monclient: authenticate success, global_id 5142 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 10 monclient: _renew_subs 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837415d1e0 con 0x7f83741503e0 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 10 client.5142.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 10 monclient: _renew_subs 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837406e030 con 0x7f83741503e0 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.193+0000 7f837ad2c640 1 librados: init done 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.194+0000 7f837ad2c640 10 client.5142.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.194+0000 7f837ad2c640 15 client.5142.objecter _session_command_op_assign -1 1 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.194+0000 7f837ad2c640 20 client.5142.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.194+0000 7f837ad2c640 10 client.5142.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.194+0000 7f837ad2c640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.196 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.194+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8374159b30 con 0x7f83741503e0 2026-02-22T23:28:36.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.207+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f835c010480 con 0x7f83741503e0 2026-02-22T23:28:36.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.207+0000 7f83697fa640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8344078e00 0x7f834407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.207+0000 7f8378aa1640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8344078e00 0x7f834407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.207+0000 7f8378aa1640 10 monclient: get_auth_request con 0x7f8344078e00 auth_method 0 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f8378aa1640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8344078e00 0x7f834407b2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8364039000 tx=0x7f8364033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 10 client.5142.objecter ms_handle_connect 0x7f8344078e00 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f835c097900 con 0x7f83741503e0 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 10 client.5142.objecter ms_dispatch2 0x7f8374000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 3 client.5142.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 3 client.5142.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 10 client.5142.objecter checking command 1 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f83440812e0 0x7f8344083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 20 client.5142.objecter _get_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 20 client.5142.objecter put_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 15 client.5142.objecter _session_command_op_remove -1 1 2026-02-22T23:28:36.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 20 client.5142.objecter _get_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 20 client.5142.objecter get_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 15 client.5142.objecter _session_command_op_assign 0 1 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 20 client.5142.objecter put_session s=0x7f8344080c10 osd=0 4 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 10 client.5142.objecter _send_command 1 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 --> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f835c0068d0 con 0x7f83440812e0 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 20 client.5142.objecter dump_active .. 0 homeless 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f835c05e4d0 con 0x7f83741503e0 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.208+0000 7f83697fa640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.209+0000 7f83792a2640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f83440812e0 0x7f8344083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.209+0000 7f83792a2640 10 monclient: get_auth_request con 0x7f83440812e0 auth_method 0 2026-02-22T23:28:36.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.210+0000 7f83792a2640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f83440812e0 0x7f8344083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.211 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.210+0000 7f83697fa640 10 client.5142.objecter ms_handle_connect 0x7f83440812e0 2026-02-22T23:28:36.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.219+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== osd.0 v2:10.20.193.6:6824/1913492598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f835c0068d0 con 0x7f83440812e0 2026-02-22T23:28:36.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.219+0000 7f83697fa640 10 client.5142.objecter ms_dispatch2 0x7f8374000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:36.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.219+0000 7f83697fa640 10 client.5142.objecter _finish_command 1 = system:0 2026-02-22T23:28:36.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.219+0000 7f83697fa640 20 client.5142.objecter put_session s=0x7f8344080c10 osd=0 4 2026-02-22T23:28:36.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.219+0000 7f83697fa640 15 client.5142.objecter _session_command_op_remove 0 1 2026-02-22T23:28:36.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 10 client.5142.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:36.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 15 client.5142.objecter _session_command_op_assign -1 2 2026-02-22T23:28:36.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 20 client.5142.objecter _get_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 20 client.5142.objecter put_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 20 client.5142.objecter _get_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 15 client.5142.objecter _session_command_op_remove -1 2 2026-02-22T23:28:36.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 20 client.5142.objecter get_session s=0x7f8344080c10 osd=0 3 2026-02-22T23:28:36.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 15 client.5142.objecter _session_command_op_assign 0 2 2026-02-22T23:28:36.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 20 client.5142.objecter put_session s=0x7f8344080c10 osd=0 4 2026-02-22T23:28:36.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 10 client.5142.objecter _send_command 2 2026-02-22T23:28:36.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.225+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 --> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f837406f5b0 con 0x7f83440812e0 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.232+0000 7f83697fa640 1 -- 10.20.193.6:0/4166733963 <== osd.0 v2:10.20.193.6:6824/1913492598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f837406f5b0 con 0x7f83440812e0 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f83697fa640 10 client.5142.objecter ms_dispatch2 0x7f8374000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f83697fa640 10 client.5142.objecter _finish_command 2 = system:0 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f83697fa640 20 client.5142.objecter put_session s=0x7f8344080c10 osd=0 4 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f83697fa640 15 client.5142.objecter _session_command_op_remove 0 2 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:36.234 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 10 librados: watch_flush enter 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 10 librados: watch_flush exit 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 10 client.5142.objecter close_session for osd.0 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f83440812e0 msgr2=0x7f8344083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:6824/1913492598,v1:10.20.193.6:6825/1913492598] conn(0x7f83440812e0 0x7f8344083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 20 client.5142.objecter put_session s=0x7f8344080c10 osd=0 1 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 20 client.5142.objecter shutdown clearing up homeless session... 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 10 client.5142.objecter successfully canceled tick 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8344078e00 msgr2=0x7f834407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8344078e00 0x7f834407b2c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8364039000 tx=0x7f8364033000 comp rx=0 tx=0).stop 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 10 monclient: shutdown 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 msgr2=0x7f837406eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 --2- 10.20.193.6:0/4166733963 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f83741503e0 0x7f837406eb10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f835c00b4d0 tx=0x7f835c00b9a0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f83792a2640 1 -- 10.20.193.6:0/4166733963 reap_dead start 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 shutdown_connections 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 >> 10.20.193.6:0/4166733963 conn(0x7f83740cd600 msgr2=0x7f8374157930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 shutdown_connections 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 -- 10.20.193.6:0/4166733963 wait complete. 2026-02-22T23:28:36.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.233+0000 7f837ad2c640 1 librados: shutdown 2026-02-22T23:28:36.242 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-22T23:28:36.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.308+0000 7f1e8082e640 10 monclient: get_monmap_and_config 2026-02-22T23:28:36.309 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.308+0000 7f1e8082e640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.308+0000 7f1e8082e640 10 monclient: monmap: 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:36.309304+0000 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:36.309304+0000 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 1 Processor -- start 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 1 -- start start 2026-02-22T23:28:36.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient: init 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient: _add_conns ranks=[3,1,2,4,5,0] 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f1e7806ff20 0x7f1e78070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient(hunting): picked mon.noname-a con 0x7f1e7806ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f1e780c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f1e7815a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e780c7e90 con 0x7f1e780c99a0 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e780c8170 con 0x7f1e7815a5e0 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f1e78159a40 con 0x7f1e7806ff20 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.309+0000 7f1e8082e640 10 monclient(hunting): authenticate will time out at 1391.036758s 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7eda4640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f1e7815a5e0 0x7f1e7814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:40528/0 (socket says 10.20.193.6:40528) 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7dda2640 1 --1- 10.20.193.6:0/2833944163 >> v1:10.20.193.40:6789/0 conn(0x7f1e780c99a0 0x7f1e78159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:47892/0 (socket says 10.20.193.6:47892) 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7eda4640 1 -- 10.20.193.6:0/2833944163 learned_addr learned my addr 10.20.193.6:0/2833944163 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 1 --2- 10.20.193.6:0/2833944163 >> v2:10.20.193.6:3300/0 conn(0x7f1e7806ff20 0x7f1e78070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 10 monclient(hunting): get_auth_request con 0x7f1e7806ff20 auth_method 0 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.310+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 1 -- 10.20.193.6:0/2833944163 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803272133 0 0) 0x7f1e780c8170 con 0x7f1e7815a5e0 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): _negotiate 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): my global_id is 5234 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 1 -- 10.20.193.6:0/2833944163 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e68003620 con 0x7f1e7815a5e0 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 1 -- 10.20.193.6:0/2833944163 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879189816 0 0) 0x7f1e780c7e90 con 0x7f1e780c99a0 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): _negotiate 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 10 monclient(hunting): my global_id is 5092 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7d5a1640 1 -- 10.20.193.6:0/2833944163 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e780c8170 con 0x7f1e780c99a0 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_done global_id 5145 payload 274 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 1 -- 10.20.193.6:0/2833944163 >> v1:10.20.193.80:6789/0 conn(0x7f1e7815a5e0 legacy=0x7f1e7814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 1 -- 10.20.193.6:0/2833944163 >> v1:10.20.193.40:6789/0 conn(0x7f1e780c99a0 legacy=0x7f1e78159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 1 monclient: found mon.noname-a 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.311+0000 7f1e7e5a3640 1 -- 10.20.193.6:0/2833944163 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e780c8450 con 0x7f1e7806ff20 2026-02-22T23:28:36.313 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.312+0000 7f1e7e5a3640 1 --2- 10.20.193.6:0/2833944163 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e78070300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1e700098d0 tx=0x7f1e7002eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71486456ca72993 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.313+0000 7f1e7d5a1640 1 -- 10.20.193.6:0/2833944163 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7003c070 con 0x7f1e7806ff20 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.313+0000 7f1e7d5a1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: dump: 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 5 monclient: authenticate success, global_id 5145 2026-02-22T23:28:36.315 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 1 -- 10.20.193.6:0/2833944163 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7002fba0 con 0x7f1e7806ff20 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 1 -- 10.20.193.6:0/2833944163 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70037440 con 0x7f1e7806ff20 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e7d5a1640 10 monclient: dump: 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 4 set_mon_vals no callback set 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 10 monclient: shutdown 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 1 -- 10.20.193.6:0/2833944163 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 msgr2=0x7f1e78070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 1 --2- 10.20.193.6:0/2833944163 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e78070300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1e700098d0 tx=0x7f1e7002eed0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 1 -- 10.20.193.6:0/2833944163 shutdown_connections 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 1 --2- 10.20.193.6:0/2833944163 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e78070300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 1 -- 10.20.193.6:0/2833944163 >> 10.20.193.6:0/2833944163 conn(0x7f1e780cd780 msgr2=0x7f1e780cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 1 -- 10.20.193.6:0/2833944163 shutdown_connections 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.314+0000 7f1e8082e640 1 -- 10.20.193.6:0/2833944163 wait complete. 2026-02-22T23:28:36.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient: monmap: 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:36.317 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 librados: starting msgr at 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 librados: starting objecter 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 Processor -- start 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 -- start start 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 librados: setting wanted keys 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 librados: calling monclient init 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient: init 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e781f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): picked mon.noname-a con 0x7f1e7806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e780c99a0 0x7f1e781f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): picked mon.noname-b con 0x7f1e780c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e7815a5e0 0x7f1e780734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): picked mon.noname-c con 0x7f1e7815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78150520 con 0x7f1e7806ff20 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780c8a10 con 0x7f1e780c99a0 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781506a0 con 0x7f1e7815a5e0 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.315+0000 7f1e8082e640 10 monclient(hunting): authenticate will time out at 1391.042806s 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e781f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.318 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e780c99a0 0x7f1e781f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e7815a5e0 0x7f1e780734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e781f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49928/0 (socket says 10.20.193.6:49928) 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 1 -- 10.20.193.6:0/69297011 learned_addr learned my addr 10.20.193.6:0/69297011 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 10 monclient(hunting): get_auth_request con 0x7f1e7806ff20 auth_method 0 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 10 monclient(hunting): get_auth_request con 0x7f1e780c99a0 auth_method 0 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 10 monclient(hunting): get_auth_request con 0x7f1e7815a5e0 auth_method 0 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7dda2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.319 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.316+0000 7f1e7eda4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 10 monclient(hunting): handle_auth_done global_id 5148 payload 824 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 1 -- 10.20.193.6:0/69297011 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e7815a5e0 msgr2=0x7f1e780734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e7815a5e0 0x7f1e780734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 1 -- 10.20.193.6:0/69297011 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e780c99a0 msgr2=0x7f1e781f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e780c99a0 0x7f1e781f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 1 monclient: found mon.noname-a 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.317+0000 7f1e7e5a3640 1 -- 10.20.193.6:0/69297011 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e781f51e0 con 0x7f1e7806ff20 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.318+0000 7f1e7e5a3640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e781f5c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009a00 tx=0x7f1e700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.318+0000 7f1e62ffd640 10 client.?.objecter ms_handle_connect 0x7f1e7806ff20 2026-02-22T23:28:36.320 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.318+0000 7f1e62ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 1 -- 10.20.193.6:0/69297011 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70046020 con 0x7f1e7806ff20 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: dump: 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.321 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 1 -- 10.20.193.6:0/69297011 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70042450 con 0x7f1e7806ff20 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 5 monclient: authenticate success, global_id 5148 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 monclient: _renew_subs 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e78157370 con 0x7f1e7806ff20 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 1 -- 10.20.193.6:0/69297011 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7003c040 con 0x7f1e7806ff20 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e7cda0640 4 set_mon_vals no callback set 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e62ffd640 10 monclient: dump: 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 client.5148.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 monclient: _renew_subs 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e78157090 con 0x7f1e7806ff20 2026-02-22T23:28:36.322 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 1 librados: init done 2026-02-22T23:28:36.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.320+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e78070300 con 0x7f1e7806ff20 2026-02-22T23:28:36.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.332+0000 7f1e62ffd640 1 -- 10.20.193.6:0/69297011 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f1e7004a3b0 con 0x7f1e7806ff20 2026-02-22T23:28:36.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.332+0000 7f1e62ffd640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e48078e50 0x7f1e4807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.332+0000 7f1e7dda2640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e48078e50 0x7f1e4807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.333 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.332+0000 7f1e7dda2640 10 monclient: get_auth_request con 0x7f1e48078e50 auth_method 0 2026-02-22T23:28:36.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.333+0000 7f1e62ffd640 10 client.5148.objecter ms_handle_connect 0x7f1e48078e50 2026-02-22T23:28:36.334 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.333+0000 7f1e7dda2640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e48078e50 0x7f1e4807b310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e781f4d50 tx=0x7f1e64007e10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.428+0000 7f1e62ffd640 1 -- 10.20.193.6:0/69297011 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f1e70082960 con 0x7f1e7806ff20 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.428+0000 7f1e62ffd640 10 client.5148.objecter ms_dispatch2 0x7f1e78000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.428+0000 7f1e62ffd640 3 client.5148.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.428+0000 7f1e62ffd640 3 client.5148.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.429+0000 7f1e62ffd640 20 client.5148.objecter dump_active .. 0 homeless 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.429+0000 7f1e62ffd640 1 -- 10.20.193.6:0/69297011 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f1e70050020 con 0x7f1e7806ff20 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.429+0000 7f1e62ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.429+0000 7f1e62ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:36.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.512+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f1e780c7200 con 0x7f1e48078e50 2026-02-22T23:28:36.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.512+0000 7f1e62ffd640 1 -- 10.20.193.6:0/69297011 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1e780c7200 con 0x7f1e48078e50 2026-02-22T23:28:36.513 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 0 to deep-scrub 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 10 librados: watch_flush enter 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 10 librados: watch_flush exit 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 20 client.5148.objecter shutdown clearing up homeless session... 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 10 client.5148.objecter successfully canceled tick 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e48078e50 msgr2=0x7f1e4807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e48078e50 0x7f1e4807b310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1e781f4d50 tx=0x7f1e64007e10 comp rx=0 tx=0).stop 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 10 monclient: shutdown 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 msgr2=0x7f1e781f5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e781f5c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009a00 tx=0x7f1e700047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 shutdown_connections 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f1e48078e50 0x7f1e4807b310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f1e7815a5e0 0x7f1e780734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f1e780c99a0 0x7f1e781f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 --2- 10.20.193.6:0/69297011 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f1e7806ff20 0x7f1e781f5c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 >> 10.20.193.6:0/69297011 conn(0x7f1e780cd780 msgr2=0x7f1e780c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 shutdown_connections 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 -- 10.20.193.6:0/69297011 wait complete. 2026-02-22T23:28:36.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.513+0000 7f1e8082e640 1 librados: shutdown 2026-02-22T23:28:36.523 INFO:tasks.ceph:Scrubbing osd.4 2026-02-22T23:28:36.523 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:36.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.592+0000 7f3c7c2ae640 10 monclient: get_monmap_and_config 2026-02-22T23:28:36.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.592+0000 7f3c7c2ae640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.592+0000 7f3c7c2ae640 10 monclient: monmap: 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:36.593319+0000 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:36.593319+0000 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 1 Processor -- start 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 1 -- start start 2026-02-22T23:28:36.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient: init 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient: _add_conns ranks=[0,2,1,3,5,4] 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3c741503e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f3c7406fdb0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f3c740c8aa0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c7415a570 con 0x7f3c741503e0 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c7415a850 con 0x7f3c740c8aa0 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c740c7fc0 con 0x7f3c7406fdb0 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.593+0000 7f3c7c2ae640 10 monclient(hunting): authenticate will time out at 1391.320702s 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.594+0000 7f3c79822640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f3c7406fdb0 0x7f3c740701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:40534/0 (socket says 10.20.193.6:40534) 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.594+0000 7f3c79822640 1 -- 10.20.193.6:0/468569906 learned_addr learned my addr 10.20.193.6:0/468569906 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:36.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.594+0000 7f3c7a023640 1 --1- 10.20.193.6:0/468569906 >> v1:10.20.193.6:6789/0 conn(0x7f3c741503e0 0x7f3c74159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35206/0 (socket says 10.20.193.6:35206) 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4055149552 0 0) 0x7f3c7415a570 con 0x7f3c741503e0 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): _negotiate 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): my global_id is 5151 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c5c003620 con 0x7f3c741503e0 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 110624208 0 0) 0x7f3c7415a850 con 0x7f3c740c8aa0 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): _negotiate 2026-02-22T23:28:36.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 10 monclient(hunting): my global_id is 5098 2026-02-22T23:28:36.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.595+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c7415a570 con 0x7f3c740c8aa0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3800981114 0 0) 0x7f3c5c003620 con 0x7f3c741503e0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 >> v1:10.20.193.80:6789/0 conn(0x7f3c7406fdb0 legacy=0x7f3c740701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 >> v1:10.20.193.40:6789/0 conn(0x7f3c740c8aa0 legacy=0x7f3c740c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c740c82a0 con 0x7f3c741503e0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f3c64002f00 con 0x7f3c741503e0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: dump: 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 >> v1:10.20.193.6:6789/0 conn(0x7f3c741503e0 legacy=0x7f3c74159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c003e00 0x7f3c5c004200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): picked mon.c con 0x7f3c5c003e00 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5c004900 0x7f3c5c004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): picked mon.b con 0x7f3c5c004900 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5c009ab0 0x7f3c5c00bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): picked mon.a con 0x7f3c5c009ab0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c00c490 con 0x7f3c5c009ab0 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c00c610 con 0x7f3c5c004900 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c00c810 con 0x7f3c5c003e00 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.597+0000 7f3c79021640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c003e00 0x7f3c5c004200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5c009ab0 0x7f3c5c00bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5c004900 0x7f3c5c004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 10 monclient(hunting): get_auth_request con 0x7f3c5c009ab0 auth_method 0 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 10 monclient(hunting): get_auth_request con 0x7f3c5c003e00 auth_method 0 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 10 monclient(hunting): get_auth_request con 0x7f3c5c004900 auth_method 0 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a824640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c7a023640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.598+0000 7f3c79822640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 10 monclient(hunting): handle_auth_done global_id 5151 payload 293 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 1 -- 10.20.193.6:0/468569906 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c003e00 msgr2=0x7f3c5c004200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c003e00 0x7f3c5c004200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 1 -- 10.20.193.6:0/468569906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5c004900 msgr2=0x7f3c5c004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c5c004900 0x7f3c5c004d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 1 monclient: found mon.a 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 1 -- 10.20.193.6:0/468569906 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c003620 con 0x7f3c5c009ab0 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7a824640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7c2ae640 5 monclient: authenticate success, global_id 5151 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.599+0000 7f3c7c2ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:36.601 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.600+0000 7f3c7a824640 1 -- 10.20.193.6:0/468569906 reap_dead start 2026-02-22T23:28:36.602 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.600+0000 7f3c7a824640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5c009ab0 0x7f3c5c00bea0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000c2f0 tx=0x7f3c7002f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79484cfd8e4dbe6d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c700038a0 con 0x7f3c5c009ab0 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 10 monclient: dump: 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c70003c40 con 0x7f3c5c009ab0 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 1 -- 10.20.193.6:0/468569906 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c700050d0 con 0x7f3c5c009ab0 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 4 set_mon_vals no callback set 2026-02-22T23:28:36.603 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c79021640 10 monclient: dump: 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.604 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 10 monclient: shutdown 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/468569906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5c009ab0 msgr2=0x7f3c5c00bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5c009ab0 0x7f3c5c00bea0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000c2f0 tx=0x7f3c7002f640 comp rx=0 tx=0).stop 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/468569906 shutdown_connections 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 1 --2- 10.20.193.6:0/468569906 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c5c009ab0 0x7f3c5c00bea0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/468569906 >> 10.20.193.6:0/468569906 conn(0x7f3c740cd600 msgr2=0x7f3c740cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/468569906 shutdown_connections 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.602+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/468569906 wait complete. 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient: monmap: 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:36.605 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 librados: starting msgr at 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 librados: starting objecter 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 Processor -- start 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 -- start start 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 librados: setting wanted keys 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 librados: calling monclient init 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient: init 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c009ab0 0x7f3c740cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): picked mon.noname-c con 0x7f3c5c009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c740cbd40 0x7f3c741f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): picked mon.noname-a con 0x7f3c740cbd40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c7415d0a0 0x7f3c7414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): picked mon.noname-b con 0x7f3c7415d0a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f3c740c7d70 con 0x7f3c740cbd40 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f3c741509d0 con 0x7f3c7415d0a0 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c740c8610 con 0x7f3c5c009ab0 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.603+0000 7f3c7c2ae640 10 monclient(hunting): authenticate will time out at 1391.330838s 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c740cbd40 0x7f3c741f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c740cbd40 0x7f3c741f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49946/0 (socket says 10.20.193.6:49946) 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 1 -- 10.20.193.6:0/2181843006 learned_addr learned my addr 10.20.193.6:0/2181843006 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 10 monclient(hunting): get_auth_request con 0x7f3c740cbd40 auth_method 0 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c009ab0 0x7f3c740cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.606 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c7415d0a0 0x7f3c7414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c79822640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 10 monclient(hunting): get_auth_request con 0x7f3c5c009ab0 auth_method 0 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 10 monclient(hunting): get_auth_request con 0x7f3c7415d0a0 auth_method 0 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.607 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a023640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.604+0000 7f3c7a824640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 10 monclient(hunting): handle_auth_done global_id 5154 payload 824 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 1 -- 10.20.193.6:0/2181843006 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c009ab0 msgr2=0x7f3c740cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f3c5c009ab0 0x7f3c740cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 1 -- 10.20.193.6:0/2181843006 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c7415d0a0 msgr2=0x7f3c7414b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3c7415d0a0 0x7f3c7414b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 1 monclient: found mon.noname-a 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.605+0000 7f3c79822640 1 -- 10.20.193.6:0/2181843006 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c74158980 con 0x7f3c740cbd40 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.606+0000 7f3c79822640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c740cbd40 0x7f3c741f4100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3c68002d40 tx=0x7f3c68004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.606+0000 7f3c5affd640 10 client.?.objecter ms_handle_connect 0x7f3c740cbd40 2026-02-22T23:28:36.608 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.606+0000 7f3c5affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800f620 con 0x7f3c740cbd40 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: dump: 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c68018070 con 0x7f3c740cbd40 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 5 monclient: authenticate success, global_id 5154 2026-02-22T23:28:36.609 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 monclient: _renew_subs 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7415d540 con 0x7f3c740cbd40 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800f9e0 con 0x7f3c740cbd40 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c78820640 4 set_mon_vals no callback set 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c5affd640 10 monclient: dump: 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 client.5154.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 monclient: _renew_subs 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7415db00 con 0x7f3c740cbd40 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 1 librados: init done 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 client.5154.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 15 client.5154.objecter _session_command_op_assign -1 1 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 20 client.5154.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 client.5154.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:36.610 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.608+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c740701d0 con 0x7f3c740cbd40 2026-02-22T23:28:36.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.619+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3c6800fb80 con 0x7f3c740cbd40 2026-02-22T23:28:36.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.619+0000 7f3c5affd640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c44078e80 0x7f3c4407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.619+0000 7f3c7a023640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c44078e80 0x7f3c4407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.619+0000 7f3c7a023640 10 monclient: get_auth_request con 0x7f3c44078e80 auth_method 0 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c7a023640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c44078e80 0x7f3c4407b340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c64002860 tx=0x7f3c640035a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 10 client.5154.objecter ms_handle_connect 0x7f3c44078e80 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f3c6805e070 con 0x7f3c740cbd40 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 10 client.5154.objecter ms_dispatch2 0x7f3c74000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 3 client.5154.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 3 client.5154.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 10 client.5154.objecter checking command 1 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7f3c44081360 0x7f3c44083780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 20 client.5154.objecter _get_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 20 client.5154.objecter put_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 15 client.5154.objecter _session_command_op_remove -1 1 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 20 client.5154.objecter _get_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 20 client.5154.objecter get_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 15 client.5154.objecter _session_command_op_assign 4 1 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 20 client.5154.objecter put_session s=0x7f3c44080c90 osd=4 4 2026-02-22T23:28:36.621 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 10 client.5154.objecter _send_command 1 2026-02-22T23:28:36.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 --> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c68026ec0 con 0x7f3c44081360 2026-02-22T23:28:36.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 20 client.5154.objecter dump_active .. 0 homeless 2026-02-22T23:28:36.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c6805e430 con 0x7f3c740cbd40 2026-02-22T23:28:36.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c5affd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:36.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.620+0000 7f3c7a824640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7f3c44081360 0x7f3c44083780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.622 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.621+0000 7f3c7a824640 10 monclient: get_auth_request con 0x7f3c44081360 auth_method 0 2026-02-22T23:28:36.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.622+0000 7f3c7a824640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7f3c44081360 0x7f3c44083780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.622+0000 7f3c5affd640 10 client.5154.objecter ms_handle_connect 0x7f3c44081360 2026-02-22T23:28:36.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.631+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== osd.4 v2:10.20.193.6:6800/2180113666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f3c68026ec0 con 0x7f3c44081360 2026-02-22T23:28:36.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.631+0000 7f3c5affd640 10 client.5154.objecter ms_dispatch2 0x7f3c74000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:36.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.631+0000 7f3c5affd640 10 client.5154.objecter _finish_command 1 = system:0 2026-02-22T23:28:36.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.631+0000 7f3c5affd640 20 client.5154.objecter put_session s=0x7f3c44080c90 osd=4 4 2026-02-22T23:28:36.632 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.631+0000 7f3c5affd640 15 client.5154.objecter _session_command_op_remove 4 1 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 10 client.5154.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 15 client.5154.objecter _session_command_op_assign -1 2 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 20 client.5154.objecter _get_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 20 client.5154.objecter put_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 20 client.5154.objecter _get_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 15 client.5154.objecter _session_command_op_remove -1 2 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 20 client.5154.objecter get_session s=0x7f3c44080c90 osd=4 3 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 15 client.5154.objecter _session_command_op_assign 4 2 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 20 client.5154.objecter put_session s=0x7f3c44080c90 osd=4 4 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 10 client.5154.objecter _send_command 2 2026-02-22T23:28:36.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.637+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 --> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3c740725b0 con 0x7f3c44081360 2026-02-22T23:28:36.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.643+0000 7f3c5affd640 1 -- 10.20.193.6:0/2181843006 <== osd.4 v2:10.20.193.6:6800/2180113666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3c740725b0 con 0x7f3c44081360 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.643+0000 7f3c5affd640 10 client.5154.objecter ms_dispatch2 0x7f3c74000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.643+0000 7f3c5affd640 10 client.5154.objecter _finish_command 2 = system:0 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.643+0000 7f3c5affd640 20 client.5154.objecter put_session s=0x7f3c44080c90 osd=4 4 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.643+0000 7f3c5affd640 15 client.5154.objecter _session_command_op_remove 4 2 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 10 librados: watch_flush enter 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 10 librados: watch_flush exit 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 10 client.5154.objecter close_session for osd.4 2026-02-22T23:28:36.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7f3c44081360 msgr2=0x7f3c44083780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:6800/2180113666,v1:10.20.193.6:6801/2180113666] conn(0x7f3c44081360 0x7f3c44083780 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 20 client.5154.objecter put_session s=0x7f3c44080c90 osd=4 1 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 20 client.5154.objecter shutdown clearing up homeless session... 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 10 client.5154.objecter successfully canceled tick 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c44078e80 msgr2=0x7f3c4407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f3c44078e80 0x7f3c4407b340 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c64002860 tx=0x7f3c640035a0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 10 monclient: shutdown 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c740cbd40 msgr2=0x7f3c741f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 --2- 10.20.193.6:0/2181843006 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f3c740cbd40 0x7f3c741f4100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3c68002d40 tx=0x7f3c68004f90 comp rx=0 tx=0).stop 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7a824640 1 -- 10.20.193.6:0/2181843006 reap_dead start 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 shutdown_connections 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 >> 10.20.193.6:0/2181843006 conn(0x7f3c740cd600 msgr2=0x7f3c7414e730 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 shutdown_connections 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 -- 10.20.193.6:0/2181843006 wait complete. 2026-02-22T23:28:36.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.644+0000 7f3c7c2ae640 1 librados: shutdown 2026-02-22T23:28:36.652 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.719+0000 7f43ff587640 10 monclient: get_monmap_and_config 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.719+0000 7f43ff587640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.719+0000 7f43ff587640 10 monclient: monmap: 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:36.719745+0000 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:36.719745+0000 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.719+0000 7f43ff587640 1 Processor -- start 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.719+0000 7f43ff587640 1 -- start start 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.719+0000 7f43ff587640 10 monclient: init 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient: _add_conns ranks=[5,1,3,2,4,0] 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f43f81527f0 0x7f43f8073530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient(hunting): picked mon.noname-c con 0x7f43f81527f0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f43f815d2c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f43f815d730 0x7f43f80ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient(hunting): picked mon.noname-a con 0x7f43f815d730 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43f8073c20 con 0x7f43f815d2c0 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f43f8074b10 con 0x7f43f815d730 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f43f8073f00 con 0x7f43f81527f0 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43ff587640 10 monclient(hunting): authenticate will time out at 1391.447437s 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43fdafd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f43f815d730 0x7f43f80ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43fd2fc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f43f81527f0 0x7f43f8073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43fcafb640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f43f815d2c0 0x7f43f815f3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:47902/0 (socket says 10.20.193.6:47902) 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.720+0000 7f43fcafb640 1 -- 10.20.193.6:0/1731303623 learned_addr learned my addr 10.20.193.6:0/1731303623 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fdafd640 10 monclient(hunting): get_auth_request con 0x7f43f815d730 auth_method 0 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fdafd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fdafd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fdafd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fd2fc640 10 monclient(hunting): get_auth_request con 0x7f43f81527f0 auth_method 0 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fd2fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fd2fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fd2fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fdafd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fdafd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fdafd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fd2fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fd2fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.721+0000 7f43fd2fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43e7fff640 1 -- 10.20.193.6:0/1731303623 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3390290826 0 0) 0x7f43f8073c20 con 0x7f43f815d2c0 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43e7fff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43e7fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43e7fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43e7fff640 10 monclient(hunting): my global_id is 5104 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43e7fff640 1 -- 10.20.193.6:0/1731303623 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43e0003620 con 0x7f43f815d2c0 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 10 monclient(hunting): handle_auth_done global_id 5246 payload 274 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 1 -- 10.20.193.6:0/1731303623 >> v2:10.20.193.6:3300/0 conn(0x7f43f815d730 msgr2=0x7f43f80ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 1 --2- 10.20.193.6:0/1731303623 >> v2:10.20.193.6:3300/0 conn(0x7f43f815d730 0x7f43f80ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 1 -- 10.20.193.6:0/1731303623 >> v1:10.20.193.40:6789/0 conn(0x7f43f815d2c0 legacy=0x7f43f815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 1 monclient: found mon.noname-c 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:36.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.722+0000 7f43fd2fc640 1 -- 10.20.193.6:0/1731303623 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f80741e0 con 0x7f43f81527f0 2026-02-22T23:28:36.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.723+0000 7f43fd2fc640 1 --2- 10.20.193.6:0/1731303623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f81527f0 0x7f43f8073530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f43f806e9a0 tx=0x7f43e802ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f27863f3b244c3 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 1 -- 10.20.193.6:0/1731303623 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e803c070 con 0x7f43f81527f0 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: dump: 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 1 -- 10.20.193.6:0/1731303623 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e802faf0 con 0x7f43f81527f0 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 1 -- 10.20.193.6:0/1731303623 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8037440 con 0x7f43f81527f0 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43e7fff640 10 monclient: dump: 2026-02-22T23:28:36.726 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43ff587640 5 monclient: authenticate success, global_id 5246 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43ff587640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.724+0000 7f43ff587640 4 set_mon_vals no callback set 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 10 monclient: shutdown 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 -- 10.20.193.6:0/1731303623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f81527f0 msgr2=0x7f43f8073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 --2- 10.20.193.6:0/1731303623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f81527f0 0x7f43f8073530 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f43f806e9a0 tx=0x7f43e802ede0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 -- 10.20.193.6:0/1731303623 shutdown_connections 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 --2- 10.20.193.6:0/1731303623 >> v2:10.20.193.6:3300/0 conn(0x7f43f815d730 0x7f43f80ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 --2- 10.20.193.6:0/1731303623 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f81527f0 0x7f43f8073530 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 -- 10.20.193.6:0/1731303623 >> 10.20.193.6:0/1731303623 conn(0x7f43f80ce730 msgr2=0x7f43f80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 -- 10.20.193.6:0/1731303623 shutdown_connections 2026-02-22T23:28:36.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 -- 10.20.193.6:0/1731303623 wait complete. 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 10 monclient: build_initial_monmap 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 10 monclient: monmap: 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 librados: starting msgr at 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 librados: starting objecter 2026-02-22T23:28:36.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 Processor -- start 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 -- start start 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 librados: setting wanted keys 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 1 librados: calling monclient init 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.725+0000 7f43ff587640 10 monclient: init 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f43f81527f0 0x7f43f8150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): picked mon.noname-b con 0x7f43f81527f0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f43f815d2c0 0x7f43f815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): picked mon.noname-a con 0x7f43f815d2c0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f815d730 0x7f43f815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): picked mon.noname-c con 0x7f43f815d730 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8074fa0 con 0x7f43f815d2c0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f43f80747a0 con 0x7f43f81527f0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8075120 con 0x7f43f815d730 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43ff587640 10 monclient(hunting): authenticate will time out at 1391.453300s 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f43f815d2c0 0x7f43f815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fdafd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f815d730 0x7f43f815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fd2fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f43f81527f0 0x7f43f8150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f43f815d2c0 0x7f43f815abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49958/0 (socket says 10.20.193.6:49958) 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 1 -- 10.20.193.6:0/605279523 learned_addr learned my addr 10.20.193.6:0/605279523 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 10 monclient(hunting): get_auth_request con 0x7f43f815d2c0 auth_method 0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fdafd640 10 monclient(hunting): get_auth_request con 0x7f43f815d730 auth_method 0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fdafd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fdafd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fdafd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fd2fc640 10 monclient(hunting): get_auth_request con 0x7f43f81527f0 auth_method 0 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fd2fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:36.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fd2fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fd2fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fcafb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.726+0000 7f43fdafd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.727+0000 7f43fdafd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.727+0000 7f43fdafd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.727+0000 7f43fd2fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.727+0000 7f43fd2fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.727+0000 7f43fd2fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 10 monclient(hunting): handle_auth_done global_id 5249 payload 824 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 1 -- 10.20.193.6:0/605279523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f43f81527f0 msgr2=0x7f43f8150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f43f81527f0 0x7f43f8150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 1 -- 10.20.193.6:0/605279523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f43f815d2c0 msgr2=0x7f43f815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f43f815d2c0 0x7f43f815abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 1 monclient: found mon.noname-c 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.728+0000 7f43fdafd640 1 -- 10.20.193.6:0/605279523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f81f7470 con 0x7f43f815d730 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.729+0000 7f43fdafd640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f815d730 0x7f43f815b100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f43f400eba0 tx=0x7f43f400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.729+0000 7f43e5ffb640 10 client.?.objecter ms_handle_connect 0x7f43f815d730 2026-02-22T23:28:36.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.729+0000 7f43e5ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 1 -- 10.20.193.6:0/605279523 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4018020 con 0x7f43f815d730 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: dump: 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.731 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 5 monclient: authenticate success, global_id 5249 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 10 monclient: _renew_subs 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f8159e00 con 0x7f43f815d730 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 10 client.5249.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 10 monclient: _renew_subs 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f815a3c0 con 0x7f43f815d730 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 1 librados: init done 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 1 -- 10.20.193.6:0/605279523 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f400cd30 con 0x7f43f815d730 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 1 -- 10.20.193.6:0/605279523 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4014680 con 0x7f43f815d730 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e5ffb640 10 monclient: dump: 2026-02-22T23:28:36.732 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43e77fe640 4 set_mon_vals no callback set 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.730+0000 7f43ff587640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.731+0000 7f43ff587640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:36.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.731+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f8152bd0 con 0x7f43f815d730 2026-02-22T23:28:36.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.742+0000 7f43e5ffb640 1 -- 10.20.193.6:0/605279523 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f43f4014820 con 0x7f43f815d730 2026-02-22T23:28:36.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.742+0000 7f43e5ffb640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f43c8078e30 0x7f43c807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:36.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.742+0000 7f43fd2fc640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f43c8078e30 0x7f43c807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:36.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.742+0000 7f43fd2fc640 10 monclient: get_auth_request con 0x7f43c8078e30 auth_method 0 2026-02-22T23:28:36.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.743+0000 7f43fd2fc640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f43c8078e30 0x7f43c807b2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f43e8039000 tx=0x7f43e8033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:36.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.743+0000 7f43e5ffb640 10 client.5249.objecter ms_handle_connect 0x7f43c8078e30 2026-02-22T23:28:36.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.743+0000 7f43e5ffb640 1 -- 10.20.193.6:0/605279523 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f43f405edd0 con 0x7f43f815d730 2026-02-22T23:28:36.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.743+0000 7f43e5ffb640 10 client.5249.objecter ms_dispatch2 0x7f43f8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:36.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.743+0000 7f43e5ffb640 3 client.5249.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:36.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.743+0000 7f43e5ffb640 3 client.5249.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:36.744 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.743+0000 7f43e5ffb640 20 client.5249.objecter dump_active .. 0 homeless 2026-02-22T23:28:36.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.840+0000 7f43e5ffb640 1 -- 10.20.193.6:0/605279523 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f43f4022030 con 0x7f43f815d730 2026-02-22T23:28:36.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.840+0000 7f43e5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:36.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.840+0000 7f43e5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:36.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.922+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f43f81501b0 con 0x7f43c8078e30 2026-02-22T23:28:36.923 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.922+0000 7f43e5ffb640 1 -- 10.20.193.6:0/605279523 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f43f81501b0 con 0x7f43c8078e30 2026-02-22T23:28:36.923 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 4 to deep-scrub 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 10 librados: watch_flush enter 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 10 librados: watch_flush exit 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 20 client.5249.objecter shutdown clearing up homeless session... 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 10 client.5249.objecter successfully canceled tick 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f43c8078e30 msgr2=0x7f43c807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f43c8078e30 0x7f43c807b2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f43e8039000 tx=0x7f43e8033000 comp rx=0 tx=0).stop 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 10 monclient: shutdown 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:36.924 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f815d730 msgr2=0x7f43f815b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f815d730 0x7f43f815b100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f43f400eba0 tx=0x7f43f400c490 comp rx=0 tx=0).stop 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 shutdown_connections 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f43c8078e30 0x7f43c807b2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f43f815d730 0x7f43f815b100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f43f815d2c0 0x7f43f815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 --2- 10.20.193.6:0/605279523 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f43f81527f0 0x7f43f8150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 >> 10.20.193.6:0/605279523 conn(0x7f43f80ce730 msgr2=0x7f43f80c8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 shutdown_connections 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 -- 10.20.193.6:0/605279523 wait complete. 2026-02-22T23:28:36.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:36.923+0000 7f43ff587640 1 librados: shutdown 2026-02-22T23:28:36.932 INFO:tasks.ceph:Scrubbing osd.8 2026-02-22T23:28:36.933 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.000+0000 7ff35b61a640 10 monclient: get_monmap_and_config 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.000+0000 7ff35b61a640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.000+0000 7ff35b61a640 10 monclient: monmap: 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:37.000812+0000 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:37.000812+0000 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:37.001 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.000+0000 7ff35b61a640 1 Processor -- start 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.000+0000 7ff35b61a640 1 -- start start 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.000+0000 7ff35b61a640 10 monclient: init 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient: _add_conns ranks=[2,5,0,1,3,4] 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7ff354155fe0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff3541563c0 0x7ff354074d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient(hunting): picked mon.noname-c con 0x7ff3541563c0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7ff3540cc310 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3540cca70 con 0x7ff3540cc310 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3540ccd50 con 0x7ff354155fe0 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff35415cf30 con 0x7ff3541563c0 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff35b61a640 10 monclient(hunting): authenticate will time out at 1391.728486s 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff359b90640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7ff3540cc310 0x7ff35415d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:35210/0 (socket says 10.20.193.6:35210) 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff359b90640 1 -- 10.20.193.6:0/1970963581 learned_addr learned my addr 10.20.193.6:0/1970963581 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.001+0000 7ff358b8e640 1 --2- 10.20.193.6:0/1970963581 >> v2:10.20.193.80:3300/0 conn(0x7ff3541563c0 0x7ff354074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.002+0000 7ff358b8e640 10 monclient(hunting): get_auth_request con 0x7ff3541563c0 auth_method 0 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.002+0000 7ff358b8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.002+0000 7ff358b8e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.002+0000 7ff358b8e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.002+0000 7ff358b8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.002+0000 7ff358b8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.002+0000 7ff358b8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff343fff640 1 -- 10.20.193.6:0/1970963581 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209788219 0 0) 0x7ff3540cca70 con 0x7ff3540cc310 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff343fff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff343fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff343fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff343fff640 10 monclient(hunting): my global_id is 5163 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff343fff640 1 -- 10.20.193.6:0/1970963581 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff33c003620 con 0x7ff3540cc310 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 10 monclient(hunting): handle_auth_done global_id 5252 payload 274 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 1 -- 10.20.193.6:0/1970963581 >> v1:10.20.193.80:6789/0 conn(0x7ff354155fe0 legacy=0x7ff35415ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 1 -- 10.20.193.6:0/1970963581 >> v1:10.20.193.6:6789/0 conn(0x7ff3540cc310 legacy=0x7ff35415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 1 monclient: found mon.noname-c 2026-02-22T23:28:37.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.003+0000 7ff358b8e640 1 -- 10.20.193.6:0/1970963581 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3540cd030 con 0x7ff3541563c0 2026-02-22T23:28:37.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.005+0000 7ff358b8e640 1 --2- 10.20.193.6:0/1970963581 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3541563c0 0x7ff354074d60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff3480098a0 tx=0x7ff34802ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1536d8d7334743 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 1 -- 10.20.193.6:0/1970963581 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34803c070 con 0x7ff3541563c0 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: dump: 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.008 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 1 -- 10.20.193.6:0/1970963581 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34802fb70 con 0x7ff3541563c0 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 1 -- 10.20.193.6:0/1970963581 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348037440 con 0x7ff3541563c0 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff35b61a640 5 monclient: authenticate success, global_id 5252 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff35b61a640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff35b61a640 4 set_mon_vals no callback set 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff343fff640 10 monclient: dump: 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.009 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff35b61a640 10 monclient: shutdown 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff35b61a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff35b61a640 1 -- 10.20.193.6:0/1970963581 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3541563c0 msgr2=0x7ff354074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.007+0000 7ff35b61a640 1 --2- 10.20.193.6:0/1970963581 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3541563c0 0x7ff354074d60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff3480098a0 tx=0x7ff34802ee60 comp rx=0 tx=0).stop 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 -- 10.20.193.6:0/1970963581 shutdown_connections 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 --2- 10.20.193.6:0/1970963581 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3541563c0 0x7ff354074d60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 -- 10.20.193.6:0/1970963581 >> 10.20.193.6:0/1970963581 conn(0x7ff3540ce830 msgr2=0x7ff3540cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 -- 10.20.193.6:0/1970963581 shutdown_connections 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 -- 10.20.193.6:0/1970963581 wait complete. 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 10 monclient: monmap: 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 librados: starting msgr at 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 librados: starting objecter 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 Processor -- start 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 -- start start 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 librados: setting wanted keys 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.008+0000 7ff35b61a640 1 librados: calling monclient init 2026-02-22T23:28:37.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient: init 2026-02-22T23:28:37.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:28:37.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3540cc310 0x7ff3541f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): picked mon.noname-c con 0x7ff3540cc310 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:37.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff354155fe0 0x7ff3540c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): picked mon.noname-b con 0x7ff354155fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff3541563c0 0x7ff354147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): picked mon.noname-a con 0x7ff3541563c0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff3540cd5f0 con 0x7ff3541563c0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff3540cd960 con 0x7ff354155fe0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff3540cc760 con 0x7ff3540cc310 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35b61a640 10 monclient(hunting): authenticate will time out at 1391.736412s 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff359b90640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff3541563c0 0x7ff354147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35938f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3540cc310 0x7ff3541f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff358b8e640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff354155fe0 0x7ff3540c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff359b90640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff3541563c0 0x7ff354147630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49970/0 (socket says 10.20.193.6:49970) 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff359b90640 1 -- 10.20.193.6:0/263173135 learned_addr learned my addr 10.20.193.6:0/263173135 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff359b90640 10 monclient(hunting): get_auth_request con 0x7ff3541563c0 auth_method 0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff359b90640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff359b90640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff359b90640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35938f640 10 monclient(hunting): get_auth_request con 0x7ff3540cc310 auth_method 0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35938f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35938f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.009+0000 7ff35938f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff358b8e640 10 monclient(hunting): get_auth_request con 0x7ff354155fe0 auth_method 0 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff358b8e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff358b8e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff358b8e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff35938f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff35938f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff35938f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff359b90640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff359b90640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff359b90640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff358b8e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff358b8e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.010+0000 7ff358b8e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 10 monclient(hunting): handle_auth_done global_id 5258 payload 824 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 1 -- 10.20.193.6:0/263173135 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff354155fe0 msgr2=0x7ff3540c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff354155fe0 0x7ff3540c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 1 -- 10.20.193.6:0/263173135 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff3541563c0 msgr2=0x7ff354147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff3541563c0 0x7ff354147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 1 monclient: found mon.noname-c 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.011+0000 7ff35938f640 1 -- 10.20.193.6:0/263173135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3540c70f0 con 0x7ff3540cc310 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.012+0000 7ff35938f640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3540cc310 0x7ff3541f7980 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c450 tx=0x7ff34400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.012+0000 7ff341ffb640 10 client.?.objecter ms_handle_connect 0x7ff3540cc310 2026-02-22T23:28:37.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.012+0000 7ff341ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3440052d0 con 0x7ff3540cc310 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: dump: 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.014 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 5 monclient: authenticate success, global_id 5258 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 monclient: _renew_subs 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3541f48f0 con 0x7ff3540cc310 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 client.5258.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 monclient: _renew_subs 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3540c5df0 con 0x7ff3540cc310 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 1 librados: init done 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff344005470 con 0x7ff3540cc310 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3440038f0 con 0x7ff3540cc310 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff341ffb640 10 monclient: dump: 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.015 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff3437fe640 4 set_mon_vals no callback set 2026-02-22T23:28:37.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 client.5258.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 15 client.5258.objecter _session_command_op_assign -1 1 2026-02-22T23:28:37.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 20 client.5258.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:37.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 client.5258.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.013+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3541557e0 con 0x7ff3540cc310 2026-02-22T23:28:37.026 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.025+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff344003a90 con 0x7ff3540cc310 2026-02-22T23:28:37.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.025+0000 7ff341ffb640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff324078e00 0x7ff32407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.025+0000 7ff358b8e640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff324078e00 0x7ff32407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.028 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.025+0000 7ff358b8e640 10 monclient: get_auth_request con 0x7ff324078e00 auth_method 0 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff358b8e640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff324078e00 0x7ff32407b2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff348039000 tx=0x7ff348033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 10 client.5258.objecter ms_handle_connect 0x7ff324078e00 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7ff3440976f0 con 0x7ff3540cc310 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 10 client.5258.objecter ms_dispatch2 0x7ff354000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 3 client.5258.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 3 client.5258.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 10 client.5258.objecter checking command 1 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7ff324081340 0x7ff324083760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 20 client.5258.objecter _get_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 20 client.5258.objecter put_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 15 client.5258.objecter _session_command_op_remove -1 1 2026-02-22T23:28:37.029 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 20 client.5258.objecter _get_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 20 client.5258.objecter get_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 15 client.5258.objecter _session_command_op_assign 8 1 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 20 client.5258.objecter put_session s=0x7ff324080c70 osd=8 4 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 10 client.5258.objecter _send_command 1 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 --> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff344015ec0 con 0x7ff324081340 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 20 client.5258.objecter dump_active .. 0 homeless 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff34405e2c0 con 0x7ff3540cc310 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff341ffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff359b90640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7ff324081340 0x7ff324083760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.030 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.026+0000 7ff359b90640 10 monclient: get_auth_request con 0x7ff324081340 auth_method 0 2026-02-22T23:28:37.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.028+0000 7ff341ffb640 10 client.5258.objecter ms_handle_connect 0x7ff324081340 2026-02-22T23:28:37.031 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.028+0000 7ff359b90640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7ff324081340 0x7ff324083760 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.037+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== osd.8 v2:10.20.193.6:6816/1002652796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7ff344015ec0 con 0x7ff324081340 2026-02-22T23:28:37.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.037+0000 7ff341ffb640 10 client.5258.objecter ms_dispatch2 0x7ff354000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:37.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.037+0000 7ff341ffb640 10 client.5258.objecter _finish_command 1 = system:0 2026-02-22T23:28:37.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.037+0000 7ff341ffb640 20 client.5258.objecter put_session s=0x7ff324080c70 osd=8 4 2026-02-22T23:28:37.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.037+0000 7ff341ffb640 15 client.5258.objecter _session_command_op_remove 8 1 2026-02-22T23:28:37.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 10 client.5258.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:37.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 15 client.5258.objecter _session_command_op_assign -1 2 2026-02-22T23:28:37.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 20 client.5258.objecter _get_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.045 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 20 client.5258.objecter put_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 20 client.5258.objecter _get_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 15 client.5258.objecter _session_command_op_remove -1 2 2026-02-22T23:28:37.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 20 client.5258.objecter get_session s=0x7ff324080c70 osd=8 3 2026-02-22T23:28:37.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 15 client.5258.objecter _session_command_op_assign 8 2 2026-02-22T23:28:37.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 20 client.5258.objecter put_session s=0x7ff324080c70 osd=8 4 2026-02-22T23:28:37.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 10 client.5258.objecter _send_command 2 2026-02-22T23:28:37.046 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.044+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 --> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff354150fa0 con 0x7ff324081340 2026-02-22T23:28:37.052 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff341ffb640 1 -- 10.20.193.6:0/263173135 <== osd.8 v2:10.20.193.6:6816/1002652796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff354150fa0 con 0x7ff324081340 2026-02-22T23:28:37.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff341ffb640 10 client.5258.objecter ms_dispatch2 0x7ff354000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:37.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff341ffb640 10 client.5258.objecter _finish_command 2 = system:0 2026-02-22T23:28:37.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff341ffb640 20 client.5258.objecter put_session s=0x7ff324080c70 osd=8 4 2026-02-22T23:28:37.053 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff341ffb640 15 client.5258.objecter _session_command_op_remove 8 2 2026-02-22T23:28:37.053 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:37.053 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:37.053 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:37.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 10 librados: watch_flush enter 2026-02-22T23:28:37.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 10 librados: watch_flush exit 2026-02-22T23:28:37.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 10 client.5258.objecter close_session for osd.8 2026-02-22T23:28:37.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7ff324081340 msgr2=0x7ff324083760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.055 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.6:6816/1002652796,v1:10.20.193.6:6817/1002652796] conn(0x7ff324081340 0x7ff324083760 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 20 client.5258.objecter put_session s=0x7ff324080c70 osd=8 1 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 20 client.5258.objecter shutdown clearing up homeless session... 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 10 client.5258.objecter successfully canceled tick 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff324078e00 msgr2=0x7ff32407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff324078e00 0x7ff32407b2c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff348039000 tx=0x7ff348033000 comp rx=0 tx=0).stop 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 10 monclient: shutdown 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3540cc310 msgr2=0x7ff3541f7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 --2- 10.20.193.6:0/263173135 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff3540cc310 0x7ff3541f7980 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c450 tx=0x7ff34400c920 comp rx=0 tx=0).stop 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff359b90640 1 -- 10.20.193.6:0/263173135 reap_dead start 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 shutdown_connections 2026-02-22T23:28:37.056 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 >> 10.20.193.6:0/263173135 conn(0x7ff3540ce830 msgr2=0x7ff35406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 shutdown_connections 2026-02-22T23:28:37.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 -- 10.20.193.6:0/263173135 wait complete. 2026-02-22T23:28:37.057 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.051+0000 7ff35b61a640 1 librados: shutdown 2026-02-22T23:28:37.062 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2026-02-22T23:28:37.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.127+0000 7f700d3a0640 10 monclient: get_monmap_and_config 2026-02-22T23:28:37.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.127+0000 7f700d3a0640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.127+0000 7f700d3a0640 10 monclient: monmap: 2026-02-22T23:28:37.128 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:37.128173+0000 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:37.128173+0000 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.127+0000 7f700d3a0640 1 Processor -- start 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 1 -- start start 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient: init 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient: _add_conns ranks=[1,0,3,2,5,4] 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f700806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f70080c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f700815a5e0 0x7f7008070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient(hunting): picked mon.noname-a con 0x7f700815a5e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:37.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70080c7e90 con 0x7f70080c99a0 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70080c8170 con 0x7f700806ff20 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f7008159a40 con 0x7f700815a5e0 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f700d3a0640 10 monclient(hunting): authenticate will time out at 1391.855558s 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.128+0000 7f70077fe640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f700815a5e0 0x7f7008070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f7006ffd640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f700806ff20 0x7f7008159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:47904/0 (socket says 10.20.193.6:47904) 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f7006ffd640 1 -- 10.20.193.6:0/2764731838 learned_addr learned my addr 10.20.193.6:0/2764731838 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f70077fe640 10 monclient(hunting): get_auth_request con 0x7f700815a5e0 auth_method 0 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f70077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f70077fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f70077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f70077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f70077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.129+0000 7f70077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f7005ffb640 1 -- 10.20.193.6:0/2764731838 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609358410 0 0) 0x7f70080c8170 con 0x7f700806ff20 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f7005ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f7005ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f7005ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f7005ffb640 10 monclient(hunting): my global_id is 5113 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f7005ffb640 1 -- 10.20.193.6:0/2764731838 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ff4003620 con 0x7f700806ff20 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 10 monclient(hunting): handle_auth_done global_id 5169 payload 274 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 1 -- 10.20.193.6:0/2764731838 >> v1:10.20.193.40:6789/0 conn(0x7f700806ff20 legacy=0x7f7008159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 1 -- 10.20.193.6:0/2764731838 >> v1:10.20.193.6:6789/0 conn(0x7f70080c99a0 legacy=0x7f700814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 1 monclient: found mon.noname-a 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.130+0000 7f70077fe640 1 -- 10.20.193.6:0/2764731838 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70080c8450 con 0x7f700815a5e0 2026-02-22T23:28:37.132 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.131+0000 7f70077fe640 1 --2- 10.20.193.6:0/2764731838 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700815a5e0 0x7f7008070300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009dc0 tx=0x7f6ff802f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46f4bd4fbfda706 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 1 -- 10.20.193.6:0/2764731838 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff803c070 con 0x7f700815a5e0 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: dump: 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.134 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 5 monclient: authenticate success, global_id 5169 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 1 -- 10.20.193.6:0/2764731838 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff8037440 con 0x7f700815a5e0 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 4 set_mon_vals no callback set 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 1 -- 10.20.193.6:0/2764731838 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8037810 con 0x7f700815a5e0 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f7005ffb640 10 monclient: dump: 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.135 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 10 monclient: shutdown 2026-02-22T23:28:37.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 1 -- 10.20.193.6:0/2764731838 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700815a5e0 msgr2=0x7f7008070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 1 --2- 10.20.193.6:0/2764731838 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700815a5e0 0x7f7008070300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009dc0 tx=0x7f6ff802f230 comp rx=0 tx=0).stop 2026-02-22T23:28:37.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 1 -- 10.20.193.6:0/2764731838 shutdown_connections 2026-02-22T23:28:37.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 1 --2- 10.20.193.6:0/2764731838 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700815a5e0 0x7f7008070300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 1 -- 10.20.193.6:0/2764731838 >> 10.20.193.6:0/2764731838 conn(0x7f70080cd780 msgr2=0x7f70080cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 1 -- 10.20.193.6:0/2764731838 shutdown_connections 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.133+0000 7f700d3a0640 1 -- 10.20.193.6:0/2764731838 wait complete. 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient: monmap: 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 librados: starting msgr at 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 librados: starting objecter 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 Processor -- start 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 -- start start 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 librados: setting wanted keys 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 librados: calling monclient init 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient: init 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700806ff20 0x7f70081f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): picked mon.noname-a con 0x7f700806ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f70080c99a0 0x7f70080bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): picked mon.noname-b con 0x7f70080c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f700815a5e0 0x7f70081f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): picked mon.noname-c con 0x7f700815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f7008150520 con 0x7f700806ff20 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f70080c8a10 con 0x7f70080c99a0 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f70081506a0 con 0x7f700815a5e0 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.134+0000 7f700d3a0640 10 monclient(hunting): authenticate will time out at 1391.861964s 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700806ff20 0x7f70081f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700806ff20 0x7f70081f1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:49986/0 (socket says 10.20.193.6:49986) 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 1 -- 10.20.193.6:0/4226334678 learned_addr learned my addr 10.20.193.6:0/4226334678 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 10 monclient(hunting): get_auth_request con 0x7f700806ff20 auth_method 0 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f700815a5e0 0x7f70081f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f7006ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 10 monclient(hunting): get_auth_request con 0x7f700815a5e0 auth_method 0 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70067fc640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f70080c99a0 0x7f70080bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.135+0000 7f70077fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f70067fc640 10 monclient(hunting): get_auth_request con 0x7f70080c99a0 auth_method 0 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f70067fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f70067fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f70067fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f70067fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f70067fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f70067fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 10 monclient(hunting): handle_auth_done global_id 5172 payload 824 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 1 -- 10.20.193.6:0/4226334678 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f700815a5e0 msgr2=0x7f70081f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f700815a5e0 0x7f70081f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 1 -- 10.20.193.6:0/4226334678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f70080c99a0 msgr2=0x7f70080bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f70080c99a0 0x7f70080bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 1 monclient: found mon.noname-a 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.136+0000 7f7006ffd640 1 -- 10.20.193.6:0/4226334678 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70081f4f40 con 0x7f700806ff20 2026-02-22T23:28:37.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.139+0000 7f7006ffd640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700806ff20 0x7f70081f1500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00c760 tx=0x7f6ffc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.139+0000 7f6fe77fe640 10 client.?.objecter ms_handle_connect 0x7f700806ff20 2026-02-22T23:28:37.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.139+0000 7f6fe77fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 1 -- 10.20.193.6:0/4226334678 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc017070 con 0x7f700806ff20 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: dump: 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 1 -- 10.20.193.6:0/4226334678 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ffc005bb0 con 0x7f700806ff20 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 5 monclient: authenticate success, global_id 5172 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 10 monclient: _renew_subs 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70081f5210 con 0x7f700806ff20 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 1 -- 10.20.193.6:0/4226334678 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc007070 con 0x7f700806ff20 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:37.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f70057fa640 4 set_mon_vals no callback set 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f6fe77fe640 10 monclient: dump: 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 10 client.5172.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 10 monclient: _renew_subs 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70081f57d0 con 0x7f700806ff20 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.140+0000 7f700d3a0640 1 librados: init done 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.141+0000 7f700d3a0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.141+0000 7f700d3a0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.141+0000 7f700d3a0640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.141+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7008070300 con 0x7f700806ff20 2026-02-22T23:28:37.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.152+0000 7f6fe77fe640 1 -- 10.20.193.6:0/4226334678 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f6ffc007210 con 0x7f700806ff20 2026-02-22T23:28:37.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.153+0000 7f6fe77fe640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6fd4078e50 0x7f6fd407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.153+0000 7f70067fc640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6fd4078e50 0x7f6fd407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.153+0000 7f70067fc640 10 monclient: get_auth_request con 0x7f6fd4078e50 auth_method 0 2026-02-22T23:28:37.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.153+0000 7f70067fc640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6fd4078e50 0x7f6fd407b310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f70081f11c0 tx=0x7f6ff00091c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.153+0000 7f6fe77fe640 10 client.5172.objecter ms_handle_connect 0x7f6fd4078e50 2026-02-22T23:28:37.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.154+0000 7f6fe77fe640 1 -- 10.20.193.6:0/4226334678 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f6ffc05d4c0 con 0x7f700806ff20 2026-02-22T23:28:37.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.154+0000 7f6fe77fe640 10 client.5172.objecter ms_dispatch2 0x7f7008000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:37.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.154+0000 7f6fe77fe640 3 client.5172.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:37.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.154+0000 7f6fe77fe640 3 client.5172.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:37.155 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.154+0000 7f6fe77fe640 20 client.5172.objecter dump_active .. 0 homeless 2026-02-22T23:28:37.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.252+0000 7f6fe77fe640 1 -- 10.20.193.6:0/4226334678 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f6ffc021020 con 0x7f700806ff20 2026-02-22T23:28:37.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.252+0000 7f6fe77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.252+0000 7f6fe77fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:37.338 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.337+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "8", "target": ["mon-mgr", ""]}) -- 0x7f70081573f0 con 0x7f6fd4078e50 2026-02-22T23:28:37.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.337+0000 7f6fe77fe640 1 -- 10.20.193.6:0/4226334678 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 8 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f70081573f0 con 0x7f6fd4078e50 2026-02-22T23:28:37.339 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 8 to deep-scrub 2026-02-22T23:28:37.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.338+0000 7f700d3a0640 10 librados: watch_flush enter 2026-02-22T23:28:37.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.338+0000 7f700d3a0640 10 librados: watch_flush exit 2026-02-22T23:28:37.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.338+0000 7f700d3a0640 20 client.5172.objecter shutdown clearing up homeless session... 2026-02-22T23:28:37.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.338+0000 7f700d3a0640 10 client.5172.objecter successfully canceled tick 2026-02-22T23:28:37.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6fd4078e50 msgr2=0x7f6fd407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6fd4078e50 0x7f6fd407b310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f70081f11c0 tx=0x7f6ff00091c0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 10 monclient: shutdown 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700806ff20 msgr2=0x7f70081f1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700806ff20 0x7f70081f1500 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00c760 tx=0x7f6ffc00cc30 comp rx=0 tx=0).stop 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 shutdown_connections 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f6fd4078e50 0x7f6fd407b310 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f700815a5e0 0x7f70081f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f70080c99a0 0x7f70080bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 --2- 10.20.193.6:0/4226334678 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f700806ff20 0x7f70081f1500 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 >> 10.20.193.6:0/4226334678 conn(0x7f70080cd780 msgr2=0x7f7008073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 shutdown_connections 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 -- 10.20.193.6:0/4226334678 wait complete. 2026-02-22T23:28:37.342 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.339+0000 7f700d3a0640 1 librados: shutdown 2026-02-22T23:28:37.349 INFO:tasks.ceph:Scrubbing osd.12 2026-02-22T23:28:37.349 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.417+0000 7f8c84914640 10 monclient: get_monmap_and_config 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.417+0000 7f8c84914640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.417+0000 7f8c84914640 10 monclient: monmap: 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:37.418298+0000 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:37.418298+0000 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.419 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.420 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:37.420 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:37.420 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:37.420 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:37.420 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:37.420 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:37.420 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 1 Processor -- start 2026-02-22T23:28:37.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 1 -- start start 2026-02-22T23:28:37.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient: init 2026-02-22T23:28:37.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient: _add_conns ranks=[2,3,1,5,0,4] 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8c7c1503e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f8c7c06fdb0 0x7f8c7c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient(hunting): picked mon.noname-a con 0x7f8c7c06fdb0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8c7c0c8aa0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c070710 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c15a570 con 0x7f8c7c1503e0 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c0c7d70 con 0x7f8c7c06fdb0 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.418+0000 7f8c84914640 10 monclient(hunting): authenticate will time out at 1392.145644s 2026-02-22T23:28:37.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c82e8a640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f8c7c0c8aa0 0x7f8c7c0c8ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:47912/0 (socket says 10.20.193.6:47912) 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c82e8a640 1 -- 10.20.193.6:0/142348758 learned_addr learned my addr 10.20.193.6:0/142348758 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 1 --2- 10.20.193.6:0/142348758 >> v2:10.20.193.6:3300/0 conn(0x7f8c7c06fdb0 0x7f8c7c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 10 monclient(hunting): get_auth_request con 0x7f8c7c06fdb0 auth_method 0 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.419+0000 7f8c81e88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 1 -- 10.20.193.6:0/142348758 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3589522662 0 0) 0x7f8c7c070710 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): _negotiate 2026-02-22T23:28:37.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): my global_id is 5119 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 1 -- 10.20.193.6:0/142348758 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c6c003620 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 1 -- 10.20.193.6:0/142348758 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2550337182 0 0) 0x7f8c7c15a570 con 0x7f8c7c1503e0 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): _negotiate 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 10 monclient(hunting): my global_id is 5264 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.420+0000 7f8c81687640 1 -- 10.20.193.6:0/142348758 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c7c070710 con 0x7f8c7c1503e0 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 10 monclient(hunting): handle_auth_done global_id 5175 payload 274 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 1 -- 10.20.193.6:0/142348758 >> v1:10.20.193.80:6789/0 conn(0x7f8c7c1503e0 legacy=0x7f8c7c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 1 -- 10.20.193.6:0/142348758 >> v1:10.20.193.40:6789/0 conn(0x7f8c7c0c8aa0 legacy=0x7f8c7c0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 1 monclient: found mon.noname-a 2026-02-22T23:28:37.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.421+0000 7f8c81e88640 1 -- 10.20.193.6:0/142348758 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c15a850 con 0x7f8c7c06fdb0 2026-02-22T23:28:37.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.422+0000 7f8c81e88640 1 --2- 10.20.193.6:0/142348758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c06fdb0 0x7f8c7c0701d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c680098a0 tx=0x7f8c6802ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6797551863928a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 1 -- 10.20.193.6:0/142348758 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6803c070 con 0x7f8c7c06fdb0 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: dump: 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.433 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 1 -- 10.20.193.6:0/142348758 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6802fc10 con 0x7f8c7c06fdb0 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 5 monclient: authenticate success, global_id 5175 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 1 -- 10.20.193.6:0/142348758 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68037440 con 0x7f8c7c06fdb0 2026-02-22T23:28:37.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 4 set_mon_vals no callback set 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c81687640 10 monclient: dump: 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.435 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 10 monclient: shutdown 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 1 -- 10.20.193.6:0/142348758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c06fdb0 msgr2=0x7f8c7c0701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 1 --2- 10.20.193.6:0/142348758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c06fdb0 0x7f8c7c0701d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c680098a0 tx=0x7f8c6802ef00 comp rx=0 tx=0).stop 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 1 -- 10.20.193.6:0/142348758 shutdown_connections 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 1 --2- 10.20.193.6:0/142348758 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c06fdb0 0x7f8c7c0701d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 1 -- 10.20.193.6:0/142348758 >> 10.20.193.6:0/142348758 conn(0x7f8c7c0cd600 msgr2=0x7f8c7c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 1 -- 10.20.193.6:0/142348758 shutdown_connections 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.424+0000 7f8c84914640 1 -- 10.20.193.6:0/142348758 wait complete. 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient: monmap: 2026-02-22T23:28:37.436 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 librados: starting msgr at 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 librados: starting objecter 2026-02-22T23:28:37.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 Processor -- start 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 -- start start 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 librados: setting wanted keys 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 librados: calling monclient init 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient: init 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8c7c06fdb0 0x7f8c7c149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): picked mon.noname-b con 0x7f8c7c06fdb0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c0c8aa0 0x7f8c7c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): picked mon.noname-a con 0x7f8c7c0c8aa0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8c7c1503e0 0x7f8c7c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): picked mon.noname-c con 0x7f8c7c1503e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:37.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c0c82a0 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c0c8610 con 0x7f8c7c06fdb0 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c15a240 con 0x7f8c7c1503e0 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c84914640 10 monclient(hunting): authenticate will time out at 1392.152774s 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.425+0000 7f8c81e88640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c0c8aa0 0x7f8c7c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8c7c06fdb0 0x7f8c7c149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c0c8aa0 0x7f8c7c15d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:50002/0 (socket says 10.20.193.6:50002) 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 1 -- 10.20.193.6:0/2622165699 learned_addr learned my addr 10.20.193.6:0/2622165699 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:37.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 10 monclient(hunting): get_auth_request con 0x7f8c7c0c8aa0 auth_method 0 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8c7c1503e0 0x7f8c7c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 10 monclient(hunting): get_auth_request con 0x7f8c7c06fdb0 auth_method 0 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c81e88640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 10 monclient(hunting): get_auth_request con 0x7f8c7c1503e0 auth_method 0 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82689640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.426+0000 7f8c82e8a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 10 monclient(hunting): handle_auth_done global_id 5178 payload 824 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 1 -- 10.20.193.6:0/2622165699 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8c7c1503e0 msgr2=0x7f8c7c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8c7c1503e0 0x7f8c7c15da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 1 -- 10.20.193.6:0/2622165699 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8c7c06fdb0 msgr2=0x7f8c7c149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8c7c06fdb0 0x7f8c7c149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 1 monclient: found mon.noname-a 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.427+0000 7f8c81e88640 1 -- 10.20.193.6:0/2622165699 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c15c160 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.428+0000 7f8c81e88640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c0c8aa0 0x7f8c7c15d4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c68016ed0 tx=0x7f8c680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.429+0000 7f8c66ffd640 10 client.?.objecter ms_handle_connect 0x7f8c7c0c8aa0 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.429+0000 7f8c66ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68046020 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:37.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: dump: 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c68042450 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6803c040 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c66ffd640 10 monclient: dump: 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.444 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c80e86640 4 set_mon_vals no callback set 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 5 monclient: authenticate success, global_id 5178 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 10 monclient: _renew_subs 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c15c440 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 10 client.5178.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 10 monclient: _renew_subs 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c15ca00 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.430+0000 7f8c84914640 1 librados: init done 2026-02-22T23:28:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.431+0000 7f8c84914640 10 client.5178.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.431+0000 7f8c84914640 15 client.5178.objecter _session_command_op_assign -1 1 2026-02-22T23:28:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.431+0000 7f8c84914640 20 client.5178.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.431+0000 7f8c84914640 10 client.5178.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.431+0000 7f8c84914640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:37.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.431+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c7c15e200 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.442+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8c6804a3b0 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.442+0000 7f8c66ffd640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8c4c078e00 0x7f8c4c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.442+0000 7f8c82689640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8c4c078e00 0x7f8c4c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.442+0000 7f8c82689640 10 monclient: get_auth_request con 0x7f8c4c078e00 auth_method 0 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c82689640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8c4c078e00 0x7f8c4c07b2c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c74003fd0 tx=0x7f8c74002a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 10 client.5178.objecter ms_handle_connect 0x7f8c4c078e00 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f8c680bc4b0 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 10 client.5178.objecter ms_dispatch2 0x7f8c7c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 3 client.5178.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 3 client.5178.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 10 client.5178.objecter checking command 1 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f8c4c0812e0 0x7f8c4c083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 20 client.5178.objecter _get_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 20 client.5178.objecter put_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 15 client.5178.objecter _session_command_op_remove -1 1 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 20 client.5178.objecter _get_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 20 client.5178.objecter get_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 15 client.5178.objecter _session_command_op_assign 12 1 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 20 client.5178.objecter put_session s=0x7f8c4c080c10 osd=12 4 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 10 client.5178.objecter _send_command 1 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 --> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c680068f0 con 0x7f8c4c0812e0 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.443+0000 7f8c66ffd640 20 client.5178.objecter dump_active .. 0 homeless 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.444+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c68083080 con 0x7f8c7c0c8aa0 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.444+0000 7f8c66ffd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.444+0000 7f8c82e8a640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f8c4c0812e0 0x7f8c4c083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.444+0000 7f8c82e8a640 10 monclient: get_auth_request con 0x7f8c4c0812e0 auth_method 0 2026-02-22T23:28:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.446+0000 7f8c66ffd640 10 client.5178.objecter ms_handle_connect 0x7f8c4c0812e0 2026-02-22T23:28:37.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.446+0000 7f8c82e8a640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f8c4c0812e0 0x7f8c4c083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.12 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.456 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.455+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== osd.12 v2:10.20.193.6:6808/138679770 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f8c680068f0 con 0x7f8c4c0812e0 2026-02-22T23:28:37.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.455+0000 7f8c66ffd640 10 client.5178.objecter ms_dispatch2 0x7f8c7c000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:37.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.455+0000 7f8c66ffd640 10 client.5178.objecter _finish_command 1 = system:0 2026-02-22T23:28:37.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.455+0000 7f8c66ffd640 20 client.5178.objecter put_session s=0x7f8c4c080c10 osd=12 4 2026-02-22T23:28:37.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.455+0000 7f8c66ffd640 15 client.5178.objecter _session_command_op_remove 12 1 2026-02-22T23:28:37.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 10 client.5178.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:37.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 15 client.5178.objecter _session_command_op_assign -1 2 2026-02-22T23:28:37.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 20 client.5178.objecter _get_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 20 client.5178.objecter put_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 20 client.5178.objecter _get_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.463 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 15 client.5178.objecter _session_command_op_remove -1 2 2026-02-22T23:28:37.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 20 client.5178.objecter get_session s=0x7f8c4c080c10 osd=12 3 2026-02-22T23:28:37.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 15 client.5178.objecter _session_command_op_assign 12 2 2026-02-22T23:28:37.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 20 client.5178.objecter put_session s=0x7f8c4c080c10 osd=12 4 2026-02-22T23:28:37.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 10 client.5178.objecter _send_command 2 2026-02-22T23:28:37.464 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.461+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 --> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8c7c14a720 con 0x7f8c4c0812e0 2026-02-22T23:28:37.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c66ffd640 1 -- 10.20.193.6:0/2622165699 <== osd.12 v2:10.20.193.6:6808/138679770 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8c7c14a720 con 0x7f8c4c0812e0 2026-02-22T23:28:37.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c66ffd640 10 client.5178.objecter ms_dispatch2 0x7f8c7c000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:37.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c66ffd640 10 client.5178.objecter _finish_command 2 = system:0 2026-02-22T23:28:37.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c66ffd640 20 client.5178.objecter put_session s=0x7f8c4c080c10 osd=12 4 2026-02-22T23:28:37.470 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c66ffd640 15 client.5178.objecter _session_command_op_remove 12 2 2026-02-22T23:28:37.470 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:37.470 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:37.471 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:37.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 10 librados: watch_flush enter 2026-02-22T23:28:37.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 10 librados: watch_flush exit 2026-02-22T23:28:37.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 10 client.5178.objecter close_session for osd.12 2026-02-22T23:28:37.472 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f8c4c0812e0 msgr2=0x7f8c4c083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:6808/138679770,v1:10.20.193.6:6809/138679770] conn(0x7f8c4c0812e0 0x7f8c4c083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 20 client.5178.objecter put_session s=0x7f8c4c080c10 osd=12 1 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 20 client.5178.objecter shutdown clearing up homeless session... 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 10 client.5178.objecter successfully canceled tick 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8c4c078e00 msgr2=0x7f8c4c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8c4c078e00 0x7f8c4c07b2c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8c74003fd0 tx=0x7f8c74002a40 comp rx=0 tx=0).stop 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 10 monclient: shutdown 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c0c8aa0 msgr2=0x7f8c7c15d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 --2- 10.20.193.6:0/2622165699 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8c7c0c8aa0 0x7f8c7c15d4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c68016ed0 tx=0x7f8c680047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.473 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c82e8a640 1 -- 10.20.193.6:0/2622165699 reap_dead start 2026-02-22T23:28:37.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 shutdown_connections 2026-02-22T23:28:37.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 >> 10.20.193.6:0/2622165699 conn(0x7f8c7c0cd600 msgr2=0x7f8c7c14f700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 shutdown_connections 2026-02-22T23:28:37.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 -- 10.20.193.6:0/2622165699 wait complete. 2026-02-22T23:28:37.474 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.468+0000 7f8c84914640 1 librados: shutdown 2026-02-22T23:28:37.478 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.547+0000 7fc5c784c640 10 monclient: get_monmap_and_config 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.547+0000 7fc5c784c640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.547+0000 7fc5c784c640 10 monclient: monmap: 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:37.548385+0000 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:37.548385+0000 2026-02-22T23:28:37.549 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:37.550 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.548+0000 7fc5c784c640 1 Processor -- start 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.548+0000 7fc5c784c640 1 -- start start 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.548+0000 7fc5c784c640 10 monclient: init 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient: _add_conns ranks=[1,5,0,3,2,4] 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fc5c01527d0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fc5c0152bb0 0x7fc5c00c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient(hunting): picked mon.noname-c con 0x7fc5c0152bb0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:37.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fc5c00c5fc0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:37.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c006e300 con 0x7fc5c00c5fc0 2026-02-22T23:28:37.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c006e5e0 con 0x7fc5c01527d0 2026-02-22T23:28:37.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c006f1f0 con 0x7fc5c0152bb0 2026-02-22T23:28:37.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.549+0000 7fc5c784c640 10 monclient(hunting): authenticate will time out at 1392.276140s 2026-02-22T23:28:37.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.571+0000 7fc5c55c1640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fc5c01527d0 0x7fc5c006da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:47926/0 (socket says 10.20.193.6:47926) 2026-02-22T23:28:37.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.571+0000 7fc5c55c1640 1 -- 10.20.193.6:0/1478137374 learned_addr learned my addr 10.20.193.6:0/1478137374 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:37.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.572+0000 7fc5c4dc0640 1 --2- 10.20.193.6:0/1478137374 >> v2:10.20.193.80:3300/0 conn(0x7fc5c0152bb0 0x7fc5c00c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.573+0000 7fc5c4dc0640 10 monclient(hunting): get_auth_request con 0x7fc5c0152bb0 auth_method 0 2026-02-22T23:28:37.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.573+0000 7fc5c4dc0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.573+0000 7fc5c4dc0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.574 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.573+0000 7fc5c4dc0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.573+0000 7fc5c4dc0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.573+0000 7fc5c4dc0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.573+0000 7fc5c4dc0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 1 -- 10.20.193.6:0/1478137374 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4127874082 0 0) 0x7fc5c006e300 con 0x7fc5c00c5fc0 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): my global_id is 5181 2026-02-22T23:28:37.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 1 -- 10.20.193.6:0/1478137374 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5a80058b0 con 0x7fc5c00c5fc0 2026-02-22T23:28:37.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 1 -- 10.20.193.6:0/1478137374 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3735946331 0 0) 0x7fc5c006e5e0 con 0x7fc5c01527d0 2026-02-22T23:28:37.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:37.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 10 monclient(hunting): my global_id is 5125 2026-02-22T23:28:37.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.574+0000 7fc5affff640 1 -- 10.20.193.6:0/1478137374 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5c006e300 con 0x7fc5c01527d0 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 10 monclient(hunting): handle_auth_done global_id 5270 payload 274 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 1 -- 10.20.193.6:0/1478137374 >> v1:10.20.193.40:6789/0 conn(0x7fc5c01527d0 legacy=0x7fc5c006da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 1 -- 10.20.193.6:0/1478137374 >> v1:10.20.193.6:6789/0 conn(0x7fc5c00c5fc0 legacy=0x7fc5c00c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 1 monclient: found mon.noname-c 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.575+0000 7fc5c4dc0640 1 -- 10.20.193.6:0/1478137374 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c006e8c0 con 0x7fc5c0152bb0 2026-02-22T23:28:37.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.576+0000 7fc5c4dc0640 1 --2- 10.20.193.6:0/1478137374 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c00c5a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40098a0 tx=0x7fc5b402ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75622e9cac3d9109 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5affff640 1 -- 10.20.193.6:0/1478137374 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b403c070 con 0x7fc5c0152bb0 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5affff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5affff640 10 monclient: dump: 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.583 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5affff640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5affff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5affff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5c784c640 5 monclient: authenticate success, global_id 5270 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.577+0000 7fc5c784c640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5affff640 1 -- 10.20.193.6:0/1478137374 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b402fb70 con 0x7fc5c0152bb0 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5affff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.584 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5affff640 1 -- 10.20.193.6:0/1478137374 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4037440 con 0x7fc5c0152bb0 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5affff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5affff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5affff640 10 monclient: dump: 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.585 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 4 set_mon_vals no callback set 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 10 monclient: shutdown 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 1 -- 10.20.193.6:0/1478137374 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 msgr2=0x7fc5c00c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 1 --2- 10.20.193.6:0/1478137374 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c00c5a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40098a0 tx=0x7fc5b402ee60 comp rx=0 tx=0).stop 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 1 -- 10.20.193.6:0/1478137374 shutdown_connections 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 1 --2- 10.20.193.6:0/1478137374 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c00c5a80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 1 -- 10.20.193.6:0/1478137374 >> 10.20.193.6:0/1478137374 conn(0x7fc5c00cd780 msgr2=0x7fc5c00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 1 -- 10.20.193.6:0/1478137374 shutdown_connections 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.578+0000 7fc5c784c640 1 -- 10.20.193.6:0/1478137374 wait complete. 2026-02-22T23:28:37.586 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient: monmap: 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 librados: starting msgr at 2026-02-22T23:28:37.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 librados: starting objecter 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 Processor -- start 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 -- start start 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 librados: setting wanted keys 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 librados: calling monclient init 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient: init 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc5c00c5fc0 0x7fc5c014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): picked mon.noname-b con 0x7fc5c00c5fc0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc5c01527d0 0x7fc5c01485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): picked mon.noname-a con 0x7fc5c01527d0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c0143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): picked mon.noname-c con 0x7fc5c0152bb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c006ee80 con 0x7fc5c01527d0 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00c6470 con 0x7fc5c00c5fc0 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00c9570 con 0x7fc5c0152bb0 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.579+0000 7fc5c784c640 10 monclient(hunting): authenticate will time out at 1392.306951s 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc5c01527d0 0x7fc5c01485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc5c00c5fc0 0x7fc5c014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c0143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc5c01527d0 0x7fc5c01485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:50008/0 (socket says 10.20.193.6:50008) 2026-02-22T23:28:37.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 1 -- 10.20.193.6:0/2997265485 learned_addr learned my addr 10.20.193.6:0/2997265485 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 10 monclient(hunting): get_auth_request con 0x7fc5c01527d0 auth_method 0 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 10 monclient(hunting): get_auth_request con 0x7fc5c0152bb0 auth_method 0 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 10 monclient(hunting): get_auth_request con 0x7fc5c00c5fc0 auth_method 0 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.590 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c4dc0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c55c1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.591 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.580+0000 7fc5c5dc2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 10 monclient(hunting): handle_auth_done global_id 5273 payload 824 2026-02-22T23:28:37.593 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 1 -- 10.20.193.6:0/2997265485 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc5c00c5fc0 msgr2=0x7fc5c014ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc5c00c5fc0 0x7fc5c014ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 1 -- 10.20.193.6:0/2997265485 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc5c01527d0 msgr2=0x7fc5c01485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc5c01527d0 0x7fc5c01485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 1 monclient: found mon.noname-c 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.582+0000 7fc5c5dc2640 1 -- 10.20.193.6:0/2997265485 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c014dce0 con 0x7fc5c0152bb0 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.583+0000 7fc5c5dc2640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c0143b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00cbf0 tx=0x7fc5bc00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.583+0000 7fc5adffb640 10 client.?.objecter ms_handle_connect 0x7fc5c0152bb0 2026-02-22T23:28:37.594 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.583+0000 7fc5adffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 1 -- 10.20.193.6:0/2997265485 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc00ce10 con 0x7fc5c0152bb0 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: dump: 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.595 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 1 -- 10.20.193.6:0/2997265485 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc003a20 con 0x7fc5c0152bb0 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 1 -- 10.20.193.6:0/2997265485 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc003e10 con 0x7fc5c0152bb0 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5adffb640 10 monclient: dump: 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.596 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5af7fe640 4 set_mon_vals no callback set 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 5 monclient: authenticate success, global_id 5273 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 10 monclient: _renew_subs 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.597 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c014dfc0 con 0x7fc5c0152bb0 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 10 client.5273.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 10 monclient: _renew_subs 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c014e580 con 0x7fc5c0152bb0 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.584+0000 7fc5c784c640 1 librados: init done 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.585+0000 7fc5c784c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.585+0000 7fc5c784c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.585+0000 7fc5c784c640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.598 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.585+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c00c5a80 con 0x7fc5c0152bb0 2026-02-22T23:28:37.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.596+0000 7fc5adffb640 1 -- 10.20.193.6:0/2997265485 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fc5bc0050d0 con 0x7fc5c0152bb0 2026-02-22T23:28:37.599 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.596+0000 7fc5adffb640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc590078e30 0x7fc59007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.596+0000 7fc5c55c1640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc590078e30 0x7fc59007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.596+0000 7fc5c55c1640 10 monclient: get_auth_request con 0x7fc590078e30 auth_method 0 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.597+0000 7fc5c55c1640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc590078e30 0x7fc59007b2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0005980 tx=0x7fc5b0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.597+0000 7fc5adffb640 10 client.5273.objecter ms_handle_connect 0x7fc590078e30 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.597+0000 7fc5adffb640 1 -- 10.20.193.6:0/2997265485 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fc5bc05e6d0 con 0x7fc5c0152bb0 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.597+0000 7fc5adffb640 10 client.5273.objecter ms_dispatch2 0x7fc5c0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.597+0000 7fc5adffb640 3 client.5273.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.597+0000 7fc5adffb640 3 client.5273.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:37.600 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.597+0000 7fc5adffb640 20 client.5273.objecter dump_active .. 0 homeless 2026-02-22T23:28:37.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.694+0000 7fc5adffb640 1 -- 10.20.193.6:0/2997265485 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fc5bc062690 con 0x7fc5c0152bb0 2026-02-22T23:28:37.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.694+0000 7fc5adffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.696 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.694+0000 7fc5adffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:37.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.777+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "12", "target": ["mon-mgr", ""]}) -- 0x7fc5c01562f0 con 0x7fc590078e30 2026-02-22T23:28:37.779 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.778+0000 7fc5adffb640 1 -- 10.20.193.6:0/2997265485 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 12 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7fc5c01562f0 con 0x7fc590078e30 2026-02-22T23:28:37.779 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 12 to deep-scrub 2026-02-22T23:28:37.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 10 librados: watch_flush enter 2026-02-22T23:28:37.781 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 10 librados: watch_flush exit 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 20 client.5273.objecter shutdown clearing up homeless session... 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 10 client.5273.objecter successfully canceled tick 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc590078e30 msgr2=0x7fc59007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc590078e30 0x7fc59007b2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0005980 tx=0x7fc5b0009040 comp rx=0 tx=0).stop 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 10 monclient: shutdown 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 msgr2=0x7fc5c0143b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c0143b00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00cbf0 tx=0x7fc5bc00a430 comp rx=0 tx=0).stop 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 shutdown_connections 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fc590078e30 0x7fc59007b2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.782 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fc5c0152bb0 0x7fc5c0143b00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fc5c01527d0 0x7fc5c01485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 --2- 10.20.193.6:0/2997265485 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fc5c00c5fc0 0x7fc5c014ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 >> 10.20.193.6:0/2997265485 conn(0x7fc5c00cd780 msgr2=0x7fc5c00ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 shutdown_connections 2026-02-22T23:28:37.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 -- 10.20.193.6:0/2997265485 wait complete. 2026-02-22T23:28:37.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.779+0000 7fc5c784c640 1 librados: shutdown 2026-02-22T23:28:37.789 INFO:tasks.ceph:Scrubbing osd.1 2026-02-22T23:28:37.789 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:37.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.856+0000 7fa569596640 10 monclient: get_monmap_and_config 2026-02-22T23:28:37.857 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.856+0000 7fa569596640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.856+0000 7fa569596640 10 monclient: monmap: 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:37.856877+0000 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:37.856877+0000 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:37.858 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:37.859 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.856+0000 7fa569596640 1 Processor -- start 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.856+0000 7fa569596640 1 -- start start 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.856+0000 7fa569596640 10 monclient: init 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient: _add_conns ranks=[5,1,4,0,2,3] 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa5641503e0 0x7fa564159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient(hunting): picked mon.noname-c con 0x7fa5641503e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa56415a070 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa5640704c0 0x7fa5640c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient(hunting): picked mon.noname-b con 0x7fa5640704c0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56415a570 con 0x7fa56415a070 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa5640c7d70 con 0x7fa5640704c0 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa5640c8610 con 0x7fa5641503e0 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa569596640 10 monclient(hunting): authenticate will time out at 1392.584229s 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa562ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa5641503e0 0x7fa564159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa5637fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa5640704c0 0x7fa5640c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa562ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa5641503e0 0x7fa564159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:57204/0 (socket says 10.20.193.6:57204) 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa562ffd640 1 -- 10.20.193.6:0/3966475853 learned_addr learned my addr 10.20.193.6:0/3966475853 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa562ffd640 10 monclient(hunting): get_auth_request con 0x7fa5641503e0 auth_method 0 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa562ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa562ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.857+0000 7fa562ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa5637fe640 10 monclient(hunting): get_auth_request con 0x7fa5640704c0 auth_method 0 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa5637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa5637fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa5637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa562ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa562ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa562ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa5637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa5637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.858+0000 7fa5637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 10 monclient(hunting): handle_auth_done global_id 5276 payload 274 2026-02-22T23:28:37.864 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 1 -- 10.20.193.6:0/3966475853 >> v2:10.20.193.40:3300/0 conn(0x7fa5640704c0 msgr2=0x7fa5640c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 1 --2- 10.20.193.6:0/3966475853 >> v2:10.20.193.40:3300/0 conn(0x7fa5640704c0 0x7fa5640c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 1 -- 10.20.193.6:0/3966475853 >> v1:10.20.193.40:6789/0 conn(0x7fa56415a070 legacy=0x7fa56406fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 1 monclient: found mon.noname-c 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.859+0000 7fa562ffd640 1 -- 10.20.193.6:0/3966475853 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56415a850 con 0x7fa5641503e0 2026-02-22T23:28:37.865 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.860+0000 7fa562ffd640 1 --2- 10.20.193.6:0/3966475853 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564159b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa5640ce490 tx=0x7fa55802ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55717ea2c69d625c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 1 -- 10.20.193.6:0/3966475853 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55803c070 con 0x7fa5641503e0 2026-02-22T23:28:37.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:37.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 10 monclient: dump: 2026-02-22T23:28:37.871 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.871 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 1 -- 10.20.193.6:0/3966475853 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55802fc70 con 0x7fa5641503e0 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.861+0000 7fa561ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa561ffb640 1 -- 10.20.193.6:0/3966475853 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558037660 con 0x7fa5641503e0 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa561ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa561ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa561ffb640 10 monclient: dump: 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.873 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 5 monclient: authenticate success, global_id 5276 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 4 set_mon_vals no callback set 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 10 monclient: shutdown 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 -- 10.20.193.6:0/3966475853 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 msgr2=0x7fa564159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 --2- 10.20.193.6:0/3966475853 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564159b30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa5640ce490 tx=0x7fa55802ef60 comp rx=0 tx=0).stop 2026-02-22T23:28:37.874 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 -- 10.20.193.6:0/3966475853 shutdown_connections 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 --2- 10.20.193.6:0/3966475853 >> v2:10.20.193.40:3300/0 conn(0x7fa5640704c0 0x7fa5640c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 --2- 10.20.193.6:0/3966475853 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564159b30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 -- 10.20.193.6:0/3966475853 >> 10.20.193.6:0/3966475853 conn(0x7fa5640cd600 msgr2=0x7fa5640cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 -- 10.20.193.6:0/3966475853 shutdown_connections 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.862+0000 7fa569596640 1 -- 10.20.193.6:0/3966475853 wait complete. 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient: monmap: 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:37.875 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 librados: starting msgr at 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 librados: starting objecter 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 Processor -- start 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 -- start start 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 librados: setting wanted keys 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 librados: calling monclient init 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient: init 2026-02-22T23:28:37.876 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5640704c0 0x7fa5641f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): picked mon.noname-a con 0x7fa5640704c0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): picked mon.noname-c con 0x7fa5641503e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa56415a070 0x7fa564073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): picked mon.noname-b con 0x7fa56415a070 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641507c0 con 0x7fa5640704c0 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640c82a0 con 0x7fa56415a070 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.877 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa564150940 con 0x7fa5641503e0 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.863+0000 7fa569596640 10 monclient(hunting): authenticate will time out at 1392.590816s 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa56415a070 0x7fa564073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5640704c0 0x7fa5641f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564143c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:57212/0 (socket says 10.20.193.6:57212) 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 1 -- 10.20.193.6:0/1755826417 learned_addr learned my addr 10.20.193.6:0/1755826417 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 10 monclient(hunting): get_auth_request con 0x7fa5641503e0 auth_method 0 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.878 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 10 monclient(hunting): get_auth_request con 0x7fa56415a070 auth_method 0 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 10 monclient(hunting): get_auth_request con 0x7fa5640704c0 auth_method 0 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5627fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.879 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa5637fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.864+0000 7fa562ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 10 monclient(hunting): handle_auth_done global_id 5279 payload 824 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 1 -- 10.20.193.6:0/1755826417 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa56415a070 msgr2=0x7fa564073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa56415a070 0x7fa564073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 1 -- 10.20.193.6:0/1755826417 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5640704c0 msgr2=0x7fa5641f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5640704c0 0x7fa5641f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 1 monclient: found mon.noname-c 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.880 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.865+0000 7fa5627fc640 1 -- 10.20.193.6:0/1755826417 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5641f7b60 con 0x7fa5641503e0 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.866+0000 7fa5627fc640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564143c70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c480 tx=0x7fa54c00c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.866+0000 7fa5437fe640 10 client.?.objecter ms_handle_connect 0x7fa5641503e0 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.866+0000 7fa5437fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c017070 con 0x7fa5641503e0 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa5437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa5437fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa5437fe640 10 monclient: dump: 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.881 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa5437fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa5437fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa5437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 5 monclient: authenticate success, global_id 5279 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 10 monclient: _renew_subs 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.882 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5640726b0 con 0x7fa5641503e0 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 10 client.5279.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 10 monclient: _renew_subs 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa564072c70 con 0x7fa5641503e0 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.867+0000 7fa569596640 1 librados: init done 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa569596640 10 client.5279.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa569596640 15 client.5279.objecter _session_command_op_assign -1 1 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa569596640 20 client.5279.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa569596640 10 client.5279.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa569596640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa564159b30 con 0x7fa5641503e0 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa564159b30 con 0x7fa5641503e0 2026-02-22T23:28:37.883 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa5437fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c021060 con 0x7fa5641503e0 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa5437fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa5437fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa5617fa640 4 set_mon_vals no callback set 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.868+0000 7fa5437fe640 10 monclient: dump: 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.884 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:37.885 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:37.885 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:37.885 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.880+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa54c0036f0 con 0x7fa5641503e0 2026-02-22T23:28:37.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.880+0000 7fa5437fe640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa530078e00 0x7fa53007b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.881+0000 7fa562ffd640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa530078e00 0x7fa53007b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.881+0000 7fa562ffd640 10 monclient: get_auth_request con 0x7fa530078e00 auth_method 0 2026-02-22T23:28:37.887 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.881+0000 7fa562ffd640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa530078e00 0x7fa53007b2c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa5640c5960 tx=0x7fa558039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.881+0000 7fa5437fe640 10 client.5279.objecter ms_handle_connect 0x7fa530078e00 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fa54c0054d0 con 0x7fa5641503e0 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 10 client.5279.objecter ms_dispatch2 0x7fa564000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 3 client.5279.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 3 client.5279.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 10 client.5279.objecter checking command 1 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7fa530081340 0x7fa530083760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 20 client.5279.objecter _get_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 20 client.5279.objecter put_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 15 client.5279.objecter _session_command_op_remove -1 1 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 20 client.5279.objecter _get_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.888 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 20 client.5279.objecter get_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 15 client.5279.objecter _session_command_op_assign 1 1 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 20 client.5279.objecter put_session s=0x7fa530080c70 osd=1 4 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 10 client.5279.objecter _send_command 1 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 --> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa54c015e70 con 0x7fa530081340 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 20 client.5279.objecter dump_active .. 0 homeless 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa54c05dc30 con 0x7fa5641503e0 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5437fe640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.882+0000 7fa5637fe640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7fa530081340 0x7fa530083760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.883+0000 7fa5637fe640 10 monclient: get_auth_request con 0x7fa530081340 auth_method 0 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.884+0000 7fa5637fe640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7fa530081340 0x7fa530083760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:37.889 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.884+0000 7fa5437fe640 10 client.5279.objecter ms_handle_connect 0x7fa530081340 2026-02-22T23:28:37.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.893+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== osd.1 v2:10.20.193.40:6824/2813004313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa54c015e70 con 0x7fa530081340 2026-02-22T23:28:37.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.893+0000 7fa5437fe640 10 client.5279.objecter ms_dispatch2 0x7fa564000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:37.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.893+0000 7fa5437fe640 10 client.5279.objecter _finish_command 1 = system:0 2026-02-22T23:28:37.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.893+0000 7fa5437fe640 20 client.5279.objecter put_session s=0x7fa530080c70 osd=1 4 2026-02-22T23:28:37.895 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.893+0000 7fa5437fe640 15 client.5279.objecter _session_command_op_remove 1 1 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 10 client.5279.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 15 client.5279.objecter _session_command_op_assign -1 2 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 20 client.5279.objecter _get_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 20 client.5279.objecter put_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 20 client.5279.objecter _get_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 15 client.5279.objecter _session_command_op_remove -1 2 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 20 client.5279.objecter get_session s=0x7fa530080c70 osd=1 3 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 15 client.5279.objecter _session_command_op_assign 1 2 2026-02-22T23:28:37.901 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 20 client.5279.objecter put_session s=0x7fa530080c70 osd=1 4 2026-02-22T23:28:37.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 10 client.5279.objecter _send_command 2 2026-02-22T23:28:37.902 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.899+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 --> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa56414ebc0 con 0x7fa530081340 2026-02-22T23:28:37.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa5437fe640 1 -- 10.20.193.6:0/1755826417 <== osd.1 v2:10.20.193.40:6824/2813004313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7fa56414ebc0 con 0x7fa530081340 2026-02-22T23:28:37.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa5437fe640 10 client.5279.objecter ms_dispatch2 0x7fa564000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:37.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa5437fe640 10 client.5279.objecter _finish_command 2 = system:0 2026-02-22T23:28:37.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa5437fe640 20 client.5279.objecter put_session s=0x7fa530080c70 osd=1 4 2026-02-22T23:28:37.915 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa5437fe640 15 client.5279.objecter _session_command_op_remove 1 2 2026-02-22T23:28:37.915 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:37.916 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T23:28:37.916 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:37.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 10 librados: watch_flush enter 2026-02-22T23:28:37.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 10 librados: watch_flush exit 2026-02-22T23:28:37.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 10 client.5279.objecter close_session for osd.1 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7fa530081340 msgr2=0x7fa530083760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.40:6824/2813004313,v1:10.20.193.40:6825/2813004313] conn(0x7fa530081340 0x7fa530083760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 20 client.5279.objecter put_session s=0x7fa530080c70 osd=1 1 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 20 client.5279.objecter shutdown clearing up homeless session... 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 10 client.5279.objecter successfully canceled tick 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa530078e00 msgr2=0x7fa53007b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa530078e00 0x7fa53007b2c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa5640c5960 tx=0x7fa558039040 comp rx=0 tx=0).stop 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 10 monclient: shutdown 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 msgr2=0x7fa564143c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 --2- 10.20.193.6:0/1755826417 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa5641503e0 0x7fa564143c70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c480 tx=0x7fa54c00c950 comp rx=0 tx=0).stop 2026-02-22T23:28:37.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa5637fe640 1 -- 10.20.193.6:0/1755826417 reap_dead start 2026-02-22T23:28:37.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 shutdown_connections 2026-02-22T23:28:37.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 >> 10.20.193.6:0/1755826417 conn(0x7fa5640cd600 msgr2=0x7fa5640cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:37.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.913+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 shutdown_connections 2026-02-22T23:28:37.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.914+0000 7fa569596640 1 -- 10.20.193.6:0/1755826417 wait complete. 2026-02-22T23:28:37.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.914+0000 7fa569596640 1 librados: shutdown 2026-02-22T23:28:37.923 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.988+0000 7f9542b03640 10 monclient: get_monmap_and_config 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.988+0000 7f9542b03640 10 monclient: build_initial_monmap 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.988+0000 7f9542b03640 10 monclient: monmap: 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:37.989278+0000 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:37.989278+0000 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:37.990 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:37.991 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:37.991 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:37.991 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:37.991 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:37.991 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:37.991 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:37.991 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:37.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 1 Processor -- start 2026-02-22T23:28:37.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 1 -- start start 2026-02-22T23:28:37.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient: init 2026-02-22T23:28:37.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:37.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient: _add_conns ranks=[4,2,1,3,0,5] 2026-02-22T23:28:37.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f953c06ff20 0x7f953c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:37.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient(hunting): picked mon.noname-b con 0x7f953c06ff20 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f953c0c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f953c15a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953c0c7e90 con 0x7f953c15a5e0 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f953c0c8170 con 0x7f953c0c99a0 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f953c159a40 con 0x7f953c06ff20 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.989+0000 7f9542b03640 10 monclient(hunting): authenticate will time out at 1392.716920s 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.990+0000 7f9540878640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f953c06ff20 0x7f953c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.990+0000 7f952bfff640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f953c0c99a0 0x7f953c159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:40554/0 (socket says 10.20.193.6:40554) 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.990+0000 7f952bfff640 1 -- 10.20.193.6:0/607671844 learned_addr learned my addr 10.20.193.6:0/607671844 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:37.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.990+0000 7f9540878640 10 monclient(hunting): get_auth_request con 0x7f953c06ff20 auth_method 0 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.990+0000 7f9540878640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.990+0000 7f9540878640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.990+0000 7f9540878640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f9540878640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f9540878640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f9540878640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f952b7fe640 1 -- 10.20.193.6:0/607671844 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032060149 0 0) 0x7f953c0c8170 con 0x7f953c0c99a0 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f952b7fe640 10 monclient(hunting): _negotiate 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f952b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f952b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f952b7fe640 10 monclient(hunting): my global_id is 5282 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.991+0000 7f952b7fe640 1 -- 10.20.193.6:0/607671844 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9524003620 con 0x7f953c0c99a0 2026-02-22T23:28:37.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 10 monclient(hunting): handle_auth_done global_id 5137 payload 274 2026-02-22T23:28:37.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 1 -- 10.20.193.6:0/607671844 >> v1:10.20.193.80:6789/0 conn(0x7f953c0c99a0 legacy=0x7f953c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 1 -- 10.20.193.6:0/607671844 >> v1:10.20.193.40:6789/0 conn(0x7f953c15a5e0 legacy=0x7f953c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:37.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 10 monclient: _finish_hunting 0 2026-02-22T23:28:37.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 1 monclient: found mon.noname-b 2026-02-22T23:28:37.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:37.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:37.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.992+0000 7f9540878640 1 -- 10.20.193.6:0/607671844 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c0c8450 con 0x7f953c06ff20 2026-02-22T23:28:38.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.994+0000 7f9540878640 1 --2- 10.20.193.6:0/607671844 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c070300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098d0 tx=0x7f952c02eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721e6c820e53a0e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 1 -- 10.20.193.6:0/607671844 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c03c070 con 0x7f953c06ff20 2026-02-22T23:28:38.002 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: dump: 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.003 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 5 monclient: authenticate success, global_id 5137 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 1 -- 10.20.193.6:0/607671844 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c02fba0 con 0x7f953c06ff20 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 1 -- 10.20.193.6:0/607671844 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c037440 con 0x7f953c06ff20 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.004 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f952b7fe640 10 monclient: dump: 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.005 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 4 set_mon_vals no callback set 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 10 monclient: shutdown 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 1 -- 10.20.193.6:0/607671844 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 msgr2=0x7f953c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 1 --2- 10.20.193.6:0/607671844 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c070300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f952c0098d0 tx=0x7f952c02eed0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 1 -- 10.20.193.6:0/607671844 shutdown_connections 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 1 --2- 10.20.193.6:0/607671844 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c070300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 1 -- 10.20.193.6:0/607671844 >> 10.20.193.6:0/607671844 conn(0x7f953c0cd780 msgr2=0x7f953c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 1 -- 10.20.193.6:0/607671844 shutdown_connections 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 1 -- 10.20.193.6:0/607671844 wait complete. 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.006 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.996+0000 7f9542b03640 10 monclient: monmap: 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 librados: starting msgr at 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 librados: starting objecter 2026-02-22T23:28:38.007 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 Processor -- start 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 -- start start 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 librados: setting wanted keys 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 librados: calling monclient init 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient: init 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c14b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): picked mon.noname-b con 0x7f953c06ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f953c0c99a0 0x7f953c06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): picked mon.noname-a con 0x7f953c0c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f953c15a5e0 0x7f953c14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): picked mon.noname-c con 0x7f953c15a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:38.008 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f953c150520 con 0x7f953c0c99a0 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f953c0c8a10 con 0x7f953c06ff20 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f953c1506a0 con 0x7f953c15a5e0 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.997+0000 7f9542b03640 10 monclient(hunting): authenticate will time out at 1392.724706s 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f953c0c99a0 0x7f953c06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f953c15a5e0 0x7f953c14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c14b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f953c15a5e0 0x7f953c14c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:57226/0 (socket says 10.20.193.6:57226) 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 1 -- 10.20.193.6:0/978366839 learned_addr learned my addr 10.20.193.6:0/978366839 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 10 monclient(hunting): get_auth_request con 0x7f953c15a5e0 auth_method 0 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 10 monclient(hunting): get_auth_request con 0x7f953c06ff20 auth_method 0 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 10 monclient(hunting): get_auth_request con 0x7f953c0c99a0 auth_method 0 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9540878640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f952bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.998+0000 7f9541079640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 10 monclient(hunting): handle_auth_done global_id 5140 payload 824 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 1 -- 10.20.193.6:0/978366839 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f953c15a5e0 msgr2=0x7f953c14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f953c15a5e0 0x7f953c14c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.011 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 1 -- 10.20.193.6:0/978366839 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f953c0c99a0 msgr2=0x7f953c06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f953c0c99a0 0x7f953c06d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 1 monclient: found mon.noname-b 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:37.999+0000 7f9540878640 1 -- 10.20.193.6:0/978366839 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953c154640 con 0x7f953c06ff20 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.000+0000 7f9540878640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c14b120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f953c072430 tx=0x7f952c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.000+0000 7f95297fa640 10 client.?.objecter ms_handle_connect 0x7f953c06ff20 2026-02-22T23:28:38.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.000+0000 7f95297fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:38.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 1 -- 10.20.193.6:0/978366839 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c03c040 con 0x7f953c06ff20 2026-02-22T23:28:38.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:38.014 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: dump: 2026-02-22T23:28:38.014 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.014 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.015 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 5 monclient: authenticate success, global_id 5140 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 monclient: _renew_subs 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953c154920 con 0x7f953c06ff20 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 client.5140.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 monclient: _renew_subs 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f953c154ee0 con 0x7f953c06ff20 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 1 librados: init done 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 1 -- 10.20.193.6:0/978366839 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c02fa60 con 0x7f953c06ff20 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f952affd640 4 set_mon_vals no callback set 2026-02-22T23:28:38.016 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 1 -- 10.20.193.6:0/978366839 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c004220 con 0x7f953c06ff20 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f95297fa640 10 monclient: dump: 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.017 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.018 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.002+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953c070300 con 0x7f953c06ff20 2026-02-22T23:28:38.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.014+0000 7f95297fa640 1 -- 10.20.193.6:0/978366839 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f952c053070 con 0x7f953c06ff20 2026-02-22T23:28:38.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.015+0000 7f95297fa640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f950c078e00 0x7f950c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.015+0000 7f952bfff640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f950c078e00 0x7f950c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.015+0000 7f952bfff640 10 monclient: get_auth_request con 0x7f950c078e00 auth_method 0 2026-02-22T23:28:38.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.015+0000 7f95297fa640 10 client.5140.objecter ms_handle_connect 0x7f950c078e00 2026-02-22T23:28:38.019 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.015+0000 7f952bfff640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f950c078e00 0x7f950c07b2c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f953c14f5b0 tx=0x7f9530007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.110+0000 7f95297fa640 1 -- 10.20.193.6:0/978366839 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f952c0bc960 con 0x7f953c06ff20 2026-02-22T23:28:38.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.110+0000 7f95297fa640 10 client.5140.objecter ms_dispatch2 0x7f953c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:38.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.110+0000 7f95297fa640 3 client.5140.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:38.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.110+0000 7f95297fa640 3 client.5140.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:38.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.110+0000 7f95297fa640 20 client.5140.objecter dump_active .. 0 homeless 2026-02-22T23:28:38.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.116+0000 7f95297fa640 1 -- 10.20.193.6:0/978366839 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f952c050020 con 0x7f953c06ff20 2026-02-22T23:28:38.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.116+0000 7f95297fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.118 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.116+0000 7f95297fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:38.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.199+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f953c1583f0 con 0x7f950c078e00 2026-02-22T23:28:38.200 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.199+0000 7f95297fa640 1 -- 10.20.193.6:0/978366839 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f953c1583f0 con 0x7f950c078e00 2026-02-22T23:28:38.200 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 1 to deep-scrub 2026-02-22T23:28:38.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 10 librados: watch_flush enter 2026-02-22T23:28:38.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 10 librados: watch_flush exit 2026-02-22T23:28:38.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 20 client.5140.objecter shutdown clearing up homeless session... 2026-02-22T23:28:38.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 10 client.5140.objecter successfully canceled tick 2026-02-22T23:28:38.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f950c078e00 msgr2=0x7f950c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f950c078e00 0x7f950c07b2c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f953c14f5b0 tx=0x7f9530007dd0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 10 monclient: shutdown 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 msgr2=0x7f953c14b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c14b120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f953c072430 tx=0x7f952c0047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 shutdown_connections 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f950c078e00 0x7f950c07b2c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f953c15a5e0 0x7f953c14c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f953c0c99a0 0x7f953c06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 --2- 10.20.193.6:0/978366839 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f953c06ff20 0x7f953c14b120 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 >> 10.20.193.6:0/978366839 conn(0x7f953c0cd780 msgr2=0x7f953c14fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.203 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 shutdown_connections 2026-02-22T23:28:38.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 -- 10.20.193.6:0/978366839 wait complete. 2026-02-22T23:28:38.204 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.200+0000 7f9542b03640 1 librados: shutdown 2026-02-22T23:28:38.209 INFO:tasks.ceph:Scrubbing osd.5 2026-02-22T23:28:38.210 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.278+0000 7f9a80133640 10 monclient: get_monmap_and_config 2026-02-22T23:28:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.278+0000 7f9a80133640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.279 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.278+0000 7f9a80133640 10 monclient: monmap: 2026-02-22T23:28:38.279 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.279 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:38.278723+0000 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:38.278723+0000 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:38.280 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.278+0000 7f9a80133640 1 Processor -- start 2026-02-22T23:28:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.278+0000 7f9a80133640 1 -- start start 2026-02-22T23:28:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.278+0000 7f9a80133640 10 monclient: init 2026-02-22T23:28:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient: _add_conns ranks=[5,4,0,1,2,3] 2026-02-22T23:28:38.283 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f9a780c9130 0x7f9a780c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient(hunting): picked mon.noname-c con 0x7f9a780c9130 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f9a7806dba0 0x7f9a780c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient(hunting): picked mon.noname-b con 0x7f9a7806dba0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f9a780c6170 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a780c65a0 con 0x7f9a780c6170 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f9a7806e3b0 con 0x7f9a7806dba0 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9a7806e690 con 0x7f9a780c9130 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a80133640 10 monclient(hunting): authenticate will time out at 1393.006114s 2026-02-22T23:28:38.284 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7dea8640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f9a780c9130 0x7f9a780c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7d6a7640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f9a7806dba0 0x7f9a780c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7e6a9640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f9a780c6170 0x7f9a7815c420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52196/0 (socket says 10.20.193.6:52196) 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7e6a9640 1 -- 10.20.193.6:0/159458455 learned_addr learned my addr 10.20.193.6:0/159458455 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7dea8640 10 monclient(hunting): get_auth_request con 0x7f9a780c9130 auth_method 0 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7dea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7dea8640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7dea8640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7d6a7640 10 monclient(hunting): get_auth_request con 0x7f9a7806dba0 auth_method 0 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7d6a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7d6a7640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7d6a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7dea8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.279+0000 7f9a7dea8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7d6a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7d6a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7d6a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 10 monclient(hunting): handle_auth_done global_id 5288 payload 274 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 1 -- 10.20.193.6:0/159458455 >> v2:10.20.193.40:3300/0 conn(0x7f9a7806dba0 msgr2=0x7f9a780c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 1 --2- 10.20.193.6:0/159458455 >> v2:10.20.193.40:3300/0 conn(0x7f9a7806dba0 0x7f9a780c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 1 -- 10.20.193.6:0/159458455 >> v1:10.20.193.6:6789/0 conn(0x7f9a780c6170 legacy=0x7f9a7815c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 1 monclient: found mon.noname-c 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.286 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.280+0000 7f9a7dea8640 1 -- 10.20.193.6:0/159458455 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7806e970 con 0x7f9a780c9130 2026-02-22T23:28:38.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.281+0000 7f9a7dea8640 1 --2- 10.20.193.6:0/159458455 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c9130 0x7f9a780c9510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9a780ce460 tx=0x7f9a6802ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed988a9a3d08a45d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 1 -- 10.20.193.6:0/159458455 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6803c070 con 0x7f9a780c9130 2026-02-22T23:28:38.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: dump: 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.294 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 1 -- 10.20.193.6:0/159458455 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6802faf0 con 0x7f9a780c9130 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 5 monclient: authenticate success, global_id 5288 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 1 -- 10.20.193.6:0/159458455 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a68037440 con 0x7f9a780c9130 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a7cea6640 10 monclient: dump: 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.295 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 4 set_mon_vals no callback set 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 10 monclient: shutdown 2026-02-22T23:28:38.296 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 1 -- 10.20.193.6:0/159458455 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c9130 msgr2=0x7f9a780c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 1 --2- 10.20.193.6:0/159458455 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c9130 0x7f9a780c9510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9a780ce460 tx=0x7f9a6802ede0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 1 -- 10.20.193.6:0/159458455 shutdown_connections 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 1 --2- 10.20.193.6:0/159458455 >> v2:10.20.193.40:3300/0 conn(0x7f9a7806dba0 0x7f9a780c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 1 --2- 10.20.193.6:0/159458455 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c9130 0x7f9a780c9510 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 1 -- 10.20.193.6:0/159458455 >> 10.20.193.6:0/159458455 conn(0x7f9a780cd600 msgr2=0x7f9a780cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.283+0000 7f9a80133640 1 -- 10.20.193.6:0/159458455 shutdown_connections 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 1 -- 10.20.193.6:0/159458455 wait complete. 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 10 monclient: monmap: 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:38.297 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 1 librados: starting msgr at 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 1 librados: starting objecter 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 1 Processor -- start 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 1 -- start start 2026-02-22T23:28:38.298 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 1 librados: setting wanted keys 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 1 librados: calling monclient init 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.284+0000 7f9a80133640 10 monclient: init 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9a7806dba0 0x7f9a781f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): picked mon.noname-b con 0x7f9a7806dba0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c6170 0x7f9a78162b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): picked mon.noname-c con 0x7f9a780c6170 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9a780c9130 0x7f9a78161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): picked mon.noname-a con 0x7f9a780c9130 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7806f210 con 0x7f9a780c9130 2026-02-22T23:28:38.299 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7806ef30 con 0x7f9a7806dba0 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7806dfe0 con 0x7f9a780c6170 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a80133640 10 monclient(hunting): authenticate will time out at 1393.012141s 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9a780c9130 0x7f9a78161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c6170 0x7f9a78162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7dea8640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9a7806dba0 0x7f9a781f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9a780c9130 0x7f9a78161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:36914/0 (socket says 10.20.193.6:36914) 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 1 -- 10.20.193.6:0/953757662 learned_addr learned my addr 10.20.193.6:0/953757662 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 10 monclient(hunting): get_auth_request con 0x7f9a780c9130 auth_method 0 2026-02-22T23:28:38.300 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 10 monclient(hunting): get_auth_request con 0x7f9a780c6170 auth_method 0 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7dea8640 10 monclient(hunting): get_auth_request con 0x7f9a7806dba0 auth_method 0 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7dea8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7dea8640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7dea8640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.301 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7e6a9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.285+0000 7f9a7d6a7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.286+0000 7f9a7dea8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.286+0000 7f9a7dea8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.286+0000 7f9a7dea8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 10 monclient(hunting): handle_auth_done global_id 5291 payload 824 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 1 -- 10.20.193.6:0/953757662 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9a7806dba0 msgr2=0x7f9a781f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f9a7806dba0 0x7f9a781f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 1 -- 10.20.193.6:0/953757662 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9a780c9130 msgr2=0x7f9a78161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f9a780c9130 0x7f9a78161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.302 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 1 monclient: found mon.noname-c 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.287+0000 7f9a7d6a7640 1 -- 10.20.193.6:0/953757662 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a78153630 con 0x7f9a780c6170 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.288+0000 7f9a7d6a7640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c6170 0x7f9a78162b70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00e8a0 tx=0x7f9a6c00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.288+0000 7f9a667fc640 10 client.?.objecter ms_handle_connect 0x7f9a780c6170 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.288+0000 7f9a667fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c017070 con 0x7f9a780c6170 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: dump: 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.303 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00c940 con 0x7f9a780c6170 2026-02-22T23:28:38.304 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00cd10 con 0x7f9a780c6170 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a667fc640 10 monclient: dump: 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.305 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a67fff640 4 set_mon_vals no callback set 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 5 monclient: authenticate success, global_id 5291 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 10 monclient: _renew_subs 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a781ee290 con 0x7f9a780c6170 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 10 client.5291.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 10 monclient: _renew_subs 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.306 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a781ed4d0 con 0x7f9a780c6170 2026-02-22T23:28:38.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.289+0000 7f9a80133640 1 librados: init done 2026-02-22T23:28:38.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.290+0000 7f9a80133640 10 client.5291.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.290+0000 7f9a80133640 15 client.5291.objecter _session_command_op_assign -1 1 2026-02-22T23:28:38.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.290+0000 7f9a80133640 20 client.5291.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:38.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.290+0000 7f9a80133640 10 client.5291.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:38.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.290+0000 7f9a80133640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.307 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.290+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a781ed280 con 0x7f9a780c6170 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.301+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f9a6c024060 con 0x7f9a780c6170 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.301+0000 7f9a667fc640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9a48078e00 0x7f9a4807b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.301+0000 7f9a7dea8640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9a48078e00 0x7f9a4807b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.302+0000 7f9a7dea8640 10 monclient: get_auth_request con 0x7f9a48078e00 auth_method 0 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.302+0000 7f9a7dea8640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9a48078e00 0x7f9a4807b2c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9a68039000 tx=0x7f9a68033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.302+0000 7f9a667fc640 10 client.5291.objecter ms_handle_connect 0x7f9a48078e00 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f9a6c0033f0 con 0x7f9a780c6170 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 10 client.5291.objecter ms_dispatch2 0x7f9a78000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 3 client.5291.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:38.310 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 3 client.5291.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 10 client.5291.objecter checking command 1 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f9a480812e0 0x7f9a48083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 20 client.5291.objecter _get_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 20 client.5291.objecter put_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 15 client.5291.objecter _session_command_op_remove -1 1 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 20 client.5291.objecter _get_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 20 client.5291.objecter get_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 15 client.5291.objecter _session_command_op_assign 5 1 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 20 client.5291.objecter put_session s=0x7f9a48080c10 osd=5 4 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 10 client.5291.objecter _send_command 1 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 --> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a48083dd0 con 0x7f9a480812e0 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 20 client.5291.objecter dump_active .. 0 homeless 2026-02-22T23:28:38.311 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9a6c05e410 con 0x7f9a780c6170 2026-02-22T23:28:38.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.303+0000 7f9a667fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:38.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.304+0000 7f9a7e6a9640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f9a480812e0 0x7f9a48083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.304+0000 7f9a7e6a9640 10 monclient: get_auth_request con 0x7f9a480812e0 auth_method 0 2026-02-22T23:28:38.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.306+0000 7f9a7e6a9640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f9a480812e0 0x7f9a48083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.312 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.306+0000 7f9a667fc640 10 client.5291.objecter ms_handle_connect 0x7f9a480812e0 2026-02-22T23:28:38.316 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.315+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== osd.5 v2:10.20.193.40:6800/2956321569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f9a48083dd0 con 0x7f9a480812e0 2026-02-22T23:28:38.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.315+0000 7f9a667fc640 10 client.5291.objecter ms_dispatch2 0x7f9a78000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:38.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.315+0000 7f9a667fc640 10 client.5291.objecter _finish_command 1 = system:0 2026-02-22T23:28:38.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.315+0000 7f9a667fc640 20 client.5291.objecter put_session s=0x7f9a48080c10 osd=5 4 2026-02-22T23:28:38.317 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.315+0000 7f9a667fc640 15 client.5291.objecter _session_command_op_remove 5 1 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 10 client.5291.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 15 client.5291.objecter _session_command_op_assign -1 2 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 20 client.5291.objecter _get_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 20 client.5291.objecter put_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 20 client.5291.objecter _get_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 15 client.5291.objecter _session_command_op_remove -1 2 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 20 client.5291.objecter get_session s=0x7f9a48080c10 osd=5 3 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 15 client.5291.objecter _session_command_op_assign 5 2 2026-02-22T23:28:38.323 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 20 client.5291.objecter put_session s=0x7f9a48080c10 osd=5 4 2026-02-22T23:28:38.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 10 client.5291.objecter _send_command 2 2026-02-22T23:28:38.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.321+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 --> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9a780c5c30 con 0x7f9a480812e0 2026-02-22T23:28:38.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a667fc640 1 -- 10.20.193.6:0/953757662 <== osd.5 v2:10.20.193.40:6800/2956321569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7f9a780c5c30 con 0x7f9a480812e0 2026-02-22T23:28:38.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a667fc640 10 client.5291.objecter ms_dispatch2 0x7f9a78000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:38.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a667fc640 10 client.5291.objecter _finish_command 2 = system:0 2026-02-22T23:28:38.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a667fc640 20 client.5291.objecter put_session s=0x7f9a48080c10 osd=5 4 2026-02-22T23:28:38.337 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a667fc640 15 client.5291.objecter _session_command_op_remove 5 2 2026-02-22T23:28:38.338 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:38.338 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T23:28:38.338 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:38.339 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 10 librados: watch_flush enter 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 10 librados: watch_flush exit 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 10 client.5291.objecter close_session for osd.5 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f9a480812e0 msgr2=0x7f9a48083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.40:6800/2956321569,v1:10.20.193.40:6801/2956321569] conn(0x7f9a480812e0 0x7f9a48083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 20 client.5291.objecter put_session s=0x7f9a48080c10 osd=5 1 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 20 client.5291.objecter shutdown clearing up homeless session... 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 10 client.5291.objecter successfully canceled tick 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9a48078e00 msgr2=0x7f9a4807b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f9a48078e00 0x7f9a4807b2c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9a68039000 tx=0x7f9a68033000 comp rx=0 tx=0).stop 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 10 monclient: shutdown 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.340 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c6170 msgr2=0x7f9a78162b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 --2- 10.20.193.6:0/953757662 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f9a780c6170 0x7f9a78162b70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00e8a0 tx=0x7f9a6c00ed70 comp rx=0 tx=0).stop 2026-02-22T23:28:38.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a7e6a9640 1 -- 10.20.193.6:0/953757662 reap_dead start 2026-02-22T23:28:38.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 shutdown_connections 2026-02-22T23:28:38.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 >> 10.20.193.6:0/953757662 conn(0x7f9a780cd600 msgr2=0x7f9a78150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 shutdown_connections 2026-02-22T23:28:38.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 -- 10.20.193.6:0/953757662 wait complete. 2026-02-22T23:28:38.341 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.336+0000 7f9a80133640 1 librados: shutdown 2026-02-22T23:28:38.345 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-22T23:28:38.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.413+0000 7fb23bf08640 10 monclient: get_monmap_and_config 2026-02-22T23:28:38.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.413+0000 7fb23bf08640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.413+0000 7fb23bf08640 10 monclient: monmap: 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:38.414034+0000 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:38.414034+0000 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:38.415 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:38.416 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:38.416 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:38.416 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:38.416 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.418 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.413+0000 7fb23bf08640 1 Processor -- start 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.413+0000 7fb23bf08640 1 -- start start 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.413+0000 7fb23bf08640 10 monclient: init 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient: _add_conns ranks=[0,5,4,1,2,3] 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb2341527d0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fb234152bb0 0x7fb2340c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient(hunting): picked mon.noname-c con 0x7fb234152bb0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fb2340c5fc0 0x7fb2340c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient(hunting): picked mon.noname-b con 0x7fb2340c5fc0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:38.419 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb23406e300 con 0x7fb2341527d0 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb23406f1f0 con 0x7fb2340c5fc0 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb2340c6470 con 0x7fb234152bb0 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23bf08640 10 monclient(hunting): authenticate will time out at 1393.141468s 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23a47e640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fb2340c5fc0 0x7fb2340c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb239c7d640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fb2341527d0 0x7fb23406da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52204/0 (socket says 10.20.193.6:52204) 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb239c7d640 1 -- 10.20.193.6:0/4152196901 learned_addr learned my addr 10.20.193.6:0/4152196901 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.414+0000 7fb23947c640 1 --2- 10.20.193.6:0/4152196901 >> v2:10.20.193.80:3300/0 conn(0x7fb234152bb0 0x7fb2340c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23947c640 10 monclient(hunting): get_auth_request con 0x7fb234152bb0 auth_method 0 2026-02-22T23:28:38.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23947c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23947c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23947c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23a47e640 10 monclient(hunting): get_auth_request con 0x7fb2340c5fc0 auth_method 0 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23a47e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23a47e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23a47e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23947c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23947c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23947c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23a47e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23a47e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.415+0000 7fb23a47e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 10 monclient(hunting): handle_auth_done global_id 5294 payload 274 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 1 -- 10.20.193.6:0/4152196901 >> v2:10.20.193.40:3300/0 conn(0x7fb2340c5fc0 msgr2=0x7fb2340c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 1 --2- 10.20.193.6:0/4152196901 >> v2:10.20.193.40:3300/0 conn(0x7fb2340c5fc0 0x7fb2340c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 1 -- 10.20.193.6:0/4152196901 >> v1:10.20.193.6:6789/0 conn(0x7fb2341527d0 legacy=0x7fb23406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 1 monclient: found mon.noname-c 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 1 -- 10.20.193.6:0/4152196901 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23406e5e0 con 0x7fb234152bb0 2026-02-22T23:28:38.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.416+0000 7fb23947c640 1 --2- 10.20.193.6:0/4152196901 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb234152bb0 0x7fb2340c5a80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb2340750e0 tx=0x7fb22802ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b18102a653c324b server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 1 -- 10.20.193.6:0/4152196901 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22803c070 con 0x7fb234152bb0 2026-02-22T23:28:38.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: dump: 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.429 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 1 -- 10.20.193.6:0/4152196901 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb22802faf0 con 0x7fb234152bb0 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 1 -- 10.20.193.6:0/4152196901 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228037440 con 0x7fb234152bb0 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 5 monclient: authenticate success, global_id 5294 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb238c7b640 10 monclient: dump: 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.430 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 4 set_mon_vals no callback set 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 10 monclient: shutdown 2026-02-22T23:28:38.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 -- 10.20.193.6:0/4152196901 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb234152bb0 msgr2=0x7fb2340c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 --2- 10.20.193.6:0/4152196901 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb234152bb0 0x7fb2340c5a80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb2340750e0 tx=0x7fb22802ede0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 -- 10.20.193.6:0/4152196901 shutdown_connections 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 --2- 10.20.193.6:0/4152196901 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb234152bb0 0x7fb2340c5a80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 --2- 10.20.193.6:0/4152196901 >> v2:10.20.193.40:3300/0 conn(0x7fb2340c5fc0 0x7fb2340c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 -- 10.20.193.6:0/4152196901 >> 10.20.193.6:0/4152196901 conn(0x7fb2340cd780 msgr2=0x7fb2340cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 -- 10.20.193.6:0/4152196901 shutdown_connections 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.418+0000 7fb23bf08640 1 -- 10.20.193.6:0/4152196901 wait complete. 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 10 monclient: monmap: 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.432 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 1 librados: starting msgr at 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 1 librados: starting objecter 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 1 Processor -- start 2026-02-22T23:28:38.433 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 1 -- start start 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 1 librados: setting wanted keys 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 1 librados: calling monclient init 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.419+0000 7fb23bf08640 10 monclient: init 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb2340c5fc0 0x7fb2340bf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): picked mon.noname-c con 0x7fb2340c5fc0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2341527d0 0x7fb234157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): picked mon.noname-b con 0x7fb2341527d0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb234152bb0 0x7fb2340bda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): picked mon.noname-a con 0x7fb234152bb0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:38.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb23406ee80 con 0x7fb234152bb0 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb23406eba0 con 0x7fb2341527d0 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340c93a0 con 0x7fb2340c5fc0 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23bf08640 10 monclient(hunting): authenticate will time out at 1393.147217s 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb234152bb0 0x7fb2340bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb239c7d640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb2340c5fc0 0x7fb2340bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23947c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2341527d0 0x7fb234157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb234152bb0 0x7fb2340bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:36916/0 (socket says 10.20.193.6:36916) 2026-02-22T23:28:38.435 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 1 -- 10.20.193.6:0/327450258 learned_addr learned my addr 10.20.193.6:0/327450258 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 10 monclient(hunting): get_auth_request con 0x7fb234152bb0 auth_method 0 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb239c7d640 10 monclient(hunting): get_auth_request con 0x7fb2340c5fc0 auth_method 0 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb239c7d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb239c7d640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb239c7d640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23947c640 10 monclient(hunting): get_auth_request con 0x7fb2341527d0 auth_method 0 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23947c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23947c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23947c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.420+0000 7fb23a47e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.421+0000 7fb239c7d640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.421+0000 7fb239c7d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.421+0000 7fb239c7d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.421+0000 7fb23947c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.421+0000 7fb23947c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.421+0000 7fb23947c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 10 monclient(hunting): handle_auth_done global_id 5297 payload 824 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 1 -- 10.20.193.6:0/327450258 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2341527d0 msgr2=0x7fb234157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2341527d0 0x7fb234157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 1 -- 10.20.193.6:0/327450258 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb234152bb0 msgr2=0x7fb2340bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb234152bb0 0x7fb2340bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 1 monclient: found mon.noname-c 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.422+0000 7fb239c7d640 1 -- 10.20.193.6:0/327450258 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2340bfcc0 con 0x7fb2340c5fc0 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.423+0000 7fb239c7d640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb2340c5fc0 0x7fb2340bf780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c500 tx=0x7fb22400c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.423+0000 7fb2227fc640 10 client.?.objecter ms_handle_connect 0x7fb2340c5fc0 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.423+0000 7fb2227fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb2227fc640 1 -- 10.20.193.6:0/327450258 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224017070 con 0x7fb2340c5fc0 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb2227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb2227fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb2227fc640 10 monclient: dump: 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb2227fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb2227fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb2227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 5 monclient: authenticate success, global_id 5297 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 monclient: _renew_subs 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2340bff30 con 0x7fb2340c5fc0 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 client.5297.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 monclient: _renew_subs 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2340c04f0 con 0x7fb2340c5fc0 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 1 librados: init done 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.424+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2340c9730 con 0x7fb2340c5fc0 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.425+0000 7fb2227fc640 1 -- 10.20.193.6:0/327450258 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb224005de0 con 0x7fb2340c5fc0 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.425+0000 7fb2227fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.425+0000 7fb2227fc640 1 -- 10.20.193.6:0/327450258 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb224007270 con 0x7fb2340c5fc0 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.425+0000 7fb2227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.425+0000 7fb2227fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.425+0000 7fb2227fc640 10 monclient: dump: 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.441 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.442 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.442 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.442 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.442 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.442 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.425+0000 7fb223fff640 4 set_mon_vals no callback set 2026-02-22T23:28:38.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.530+0000 7fb2227fc640 1 -- 10.20.193.6:0/327450258 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb224007410 con 0x7fb2340c5fc0 2026-02-22T23:28:38.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.530+0000 7fb2227fc640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb204078e00 0x7fb20407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.530+0000 7fb23947c640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb204078e00 0x7fb20407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.530+0000 7fb23947c640 10 monclient: get_auth_request con 0x7fb204078e00 auth_method 0 2026-02-22T23:28:38.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.531+0000 7fb2227fc640 10 client.5297.objecter ms_handle_connect 0x7fb204078e00 2026-02-22T23:28:38.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.531+0000 7fb23947c640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb204078e00 0x7fb20407b2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb228039000 tx=0x7fb228033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.531+0000 7fb2227fc640 1 -- 10.20.193.6:0/327450258 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fb224007730 con 0x7fb2340c5fc0 2026-02-22T23:28:38.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.532+0000 7fb2227fc640 10 client.5297.objecter ms_dispatch2 0x7fb234000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:38.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.532+0000 7fb2227fc640 3 client.5297.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:38.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.532+0000 7fb2227fc640 3 client.5297.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:38.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.532+0000 7fb2227fc640 20 client.5297.objecter dump_active .. 0 homeless 2026-02-22T23:28:38.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.538+0000 7fb2227fc640 1 -- 10.20.193.6:0/327450258 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb224022020 con 0x7fb2340c5fc0 2026-02-22T23:28:38.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.538+0000 7fb2227fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.538+0000 7fb2227fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:38.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.623+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb2340c77d0 con 0x7fb204078e00 2026-02-22T23:28:38.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.623+0000 7fb2227fc640 1 -- 10.20.193.6:0/327450258 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb2340c77d0 con 0x7fb204078e00 2026-02-22T23:28:38.625 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 5 to deep-scrub 2026-02-22T23:28:38.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 10 librados: watch_flush enter 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 10 librados: watch_flush exit 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 20 client.5297.objecter shutdown clearing up homeless session... 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 10 client.5297.objecter successfully canceled tick 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb204078e00 msgr2=0x7fb20407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb204078e00 0x7fb20407b2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb228039000 tx=0x7fb228033000 comp rx=0 tx=0).stop 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 10 monclient: shutdown 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb2340c5fc0 msgr2=0x7fb2340bf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb2340c5fc0 0x7fb2340bf780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb22400c500 tx=0x7fb22400c9d0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.627 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 shutdown_connections 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb204078e00 0x7fb20407b2c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb234152bb0 0x7fb2340bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2341527d0 0x7fb234157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 --2- 10.20.193.6:0/327450258 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb2340c5fc0 0x7fb2340bf780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 >> 10.20.193.6:0/327450258 conn(0x7fb2340cd780 msgr2=0x7fb234150180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 shutdown_connections 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 -- 10.20.193.6:0/327450258 wait complete. 2026-02-22T23:28:38.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.624+0000 7fb23bf08640 1 librados: shutdown 2026-02-22T23:28:38.634 INFO:tasks.ceph:Scrubbing osd.9 2026-02-22T23:28:38.635 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.703+0000 7fd77af53640 10 monclient: get_monmap_and_config 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.703+0000 7fd77af53640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.703+0000 7fd77af53640 10 monclient: monmap: 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:38.704346+0000 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:38.704346+0000 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.705 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.706 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:38.706 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:38.706 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:38.706 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:38.706 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:38.706 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:38.706 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 1 Processor -- start 2026-02-22T23:28:38.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 1 -- start start 2026-02-22T23:28:38.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient: init 2026-02-22T23:28:38.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient: _add_conns ranks=[2,4,5,3,1,0] 2026-02-22T23:28:38.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd7741503e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fd77406fdb0 0x7fd7740701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient(hunting): picked mon.noname-b con 0x7fd77406fdb0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd7740c8aa0 0x7fd7740c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient(hunting): picked mon.noname-c con 0x7fd7740c8aa0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd774070710 con 0x7fd7741503e0 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd7740c7d70 con 0x7fd77406fdb0 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd7740c8610 con 0x7fd7740c8aa0 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.704+0000 7fd77af53640 10 monclient(hunting): authenticate will time out at 1393.431743s 2026-02-22T23:28:38.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fd77406fdb0 0x7fd7740701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd778cc8640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fd7741503e0 0x7fd774159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:46646/0 (socket says 10.20.193.6:46646) 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd778cc8640 1 -- 10.20.193.6:0/139742090 learned_addr learned my addr 10.20.193.6:0/139742090 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 10 monclient(hunting): get_auth_request con 0x7fd77406fdb0 auth_method 0 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd7794c9640 1 --2- 10.20.193.6:0/139742090 >> v2:10.20.193.80:3300/0 conn(0x7fd7740c8aa0 0x7fd7740c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.705+0000 7fd76bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd7794c9640 10 monclient(hunting): get_auth_request con 0x7fd7740c8aa0 auth_method 0 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd7794c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd7794c9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd7794c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 10 monclient(hunting): handle_auth_done global_id 5155 payload 274 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 1 -- 10.20.193.6:0/139742090 >> v2:10.20.193.80:3300/0 conn(0x7fd7740c8aa0 msgr2=0x7fd7740c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 1 --2- 10.20.193.6:0/139742090 >> v2:10.20.193.80:3300/0 conn(0x7fd7740c8aa0 0x7fd7740c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 1 -- 10.20.193.6:0/139742090 >> v1:10.20.193.80:6789/0 conn(0x7fd7741503e0 legacy=0x7fd774159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 1 monclient: found mon.noname-b 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.706+0000 7fd76bfff640 1 -- 10.20.193.6:0/139742090 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7740c9420 con 0x7fd77406fdb0 2026-02-22T23:28:38.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.707+0000 7fd76bfff640 1 --2- 10.20.193.6:0/139742090 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd77406fdb0 0x7fd7740701d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7740709b0 tx=0x7fd75c02eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4524b657c86d1e server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 1 -- 10.20.193.6:0/139742090 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c03c070 con 0x7fd77406fdb0 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: dump: 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.719 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 1 -- 10.20.193.6:0/139742090 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75c02fb80 con 0x7fd77406fdb0 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 1 -- 10.20.193.6:0/139742090 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c037440 con 0x7fd77406fdb0 2026-02-22T23:28:38.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 5 monclient: authenticate success, global_id 5155 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 4 set_mon_vals no callback set 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd76b7fe640 10 monclient: dump: 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.721 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 10 monclient: shutdown 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 -- 10.20.193.6:0/139742090 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd77406fdb0 msgr2=0x7fd7740701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 --2- 10.20.193.6:0/139742090 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd77406fdb0 0x7fd7740701d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7740709b0 tx=0x7fd75c02eeb0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 -- 10.20.193.6:0/139742090 shutdown_connections 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 --2- 10.20.193.6:0/139742090 >> v2:10.20.193.80:3300/0 conn(0x7fd7740c8aa0 0x7fd7740c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 --2- 10.20.193.6:0/139742090 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd77406fdb0 0x7fd7740701d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 -- 10.20.193.6:0/139742090 >> 10.20.193.6:0/139742090 conn(0x7fd7740cd600 msgr2=0x7fd7740cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 -- 10.20.193.6:0/139742090 shutdown_connections 2026-02-22T23:28:38.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.709+0000 7fd77af53640 1 -- 10.20.193.6:0/139742090 wait complete. 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 10 monclient: monmap: 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 1 librados: starting msgr at 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 1 librados: starting objecter 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 1 Processor -- start 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 1 -- start start 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 1 librados: setting wanted keys 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 1 librados: calling monclient init 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.710+0000 7fd77af53640 10 monclient: init 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd77406fdb0 0x7fd7741e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): picked mon.noname-c con 0x7fd77406fdb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd7740c8aa0 0x7fd7741e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): picked mon.noname-a con 0x7fd7740c8aa0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd7741503e0 0x7fd77414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): picked mon.noname-b con 0x7fd7741503e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd7740c7fc0 con 0x7fd7740c8aa0 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd77415a850 con 0x7fd7741503e0 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd7740c82a0 con 0x7fd77406fdb0 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd77af53640 10 monclient(hunting): authenticate will time out at 1393.438082s 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd77406fdb0 0x7fd7741e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd7741503e0 0x7fd77414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd7740c8aa0 0x7fd7741e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd77406fdb0 0x7fd7741e68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43080/0 (socket says 10.20.193.6:43080) 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 1 -- 10.20.193.6:0/917934783 learned_addr learned my addr 10.20.193.6:0/917934783 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 10 monclient(hunting): get_auth_request con 0x7fd77406fdb0 auth_method 0 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 10 monclient(hunting): get_auth_request con 0x7fd7741503e0 auth_method 0 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 10 monclient(hunting): get_auth_request con 0x7fd7740c8aa0 auth_method 0 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd778cc8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd7794c9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.711+0000 7fd76bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 10 monclient(hunting): handle_auth_done global_id 5158 payload 824 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 1 -- 10.20.193.6:0/917934783 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd77406fdb0 msgr2=0x7fd7741e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd77406fdb0 0x7fd7741e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 1 -- 10.20.193.6:0/917934783 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd7740c8aa0 msgr2=0x7fd7741e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd7740c8aa0 0x7fd7741e6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 1 monclient: found mon.noname-b 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.713+0000 7fd7794c9640 1 -- 10.20.193.6:0/917934783 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7741e6150 con 0x7fd7741503e0 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7794c9640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd7741503e0 0x7fd77414b840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd770007d90 tx=0x7fd77000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 10 client.?.objecter ms_handle_connect 0x7fd7741503e0 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77000ede0 con 0x7fd7741503e0 2026-02-22T23:28:38.728 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 10 monclient: dump: 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd7697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 5 monclient: authenticate success, global_id 5158 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 10 monclient: _renew_subs 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7741550a0 con 0x7fd7741503e0 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 10 client.5158.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 10 monclient: _renew_subs 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd774155660 con 0x7fd7741503e0 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.714+0000 7fd77af53640 1 librados: init done 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd77af53640 10 client.5158.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd77af53640 15 client.5158.objecter _session_command_op_assign -1 1 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd77af53640 20 client.5158.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd77af53640 10 client.5158.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd77af53640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7741507c0 con 0x7fd7741503e0 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7741507c0 con 0x7fd7741503e0 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd7697fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770003c90 con 0x7fd7741503e0 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd7697fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd7697fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd7697fa640 10 monclient: dump: 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.731 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.715+0000 7fd76affd640 4 set_mon_vals no callback set 2026-02-22T23:28:38.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.728+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd770004020 con 0x7fd7741503e0 2026-02-22T23:28:38.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd744078e30 0x7fd74407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd778cc8640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd744078e30 0x7fd74407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd778cc8640 10 monclient: get_auth_request con 0x7fd744078e30 auth_method 0 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fd770098230 con 0x7fd7741503e0 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 10 client.5158.objecter ms_dispatch2 0x7fd774000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 3 client.5158.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 3 client.5158.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 10 client.5158.objecter checking command 1 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd778cc8640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd744078e30 0x7fd74407b2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd764002770 tx=0x7fd764009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7fd744081370 0x7fd744083790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 20 client.5158.objecter _get_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 20 client.5158.objecter put_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 15 client.5158.objecter _session_command_op_remove -1 1 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 20 client.5158.objecter _get_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 20 client.5158.objecter get_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 15 client.5158.objecter _session_command_op_assign 9 1 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 20 client.5158.objecter put_session s=0x7fd744080ca0 osd=9 4 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 10 client.5158.objecter _send_command 1 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 --> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd744083e60 con 0x7fd744081370 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 20 client.5158.objecter dump_active .. 0 homeless 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 10 client.5158.objecter ms_handle_connect 0x7fd744078e30 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd77005ecf0 con 0x7fd7741503e0 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.729+0000 7fd7697fa640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.730+0000 7fd76bfff640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7fd744081370 0x7fd744083790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.730+0000 7fd76bfff640 10 monclient: get_auth_request con 0x7fd744081370 auth_method 0 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.731+0000 7fd7697fa640 10 client.5158.objecter ms_handle_connect 0x7fd744081370 2026-02-22T23:28:38.736 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.731+0000 7fd76bfff640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7fd744081370 0x7fd744083790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.741+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== osd.9 v2:10.20.193.40:6808/979607499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd744083e60 con 0x7fd744081370 2026-02-22T23:28:38.742 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.741+0000 7fd7697fa640 10 client.5158.objecter ms_dispatch2 0x7fd774000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:38.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.741+0000 7fd7697fa640 10 client.5158.objecter _finish_command 1 = system:0 2026-02-22T23:28:38.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.741+0000 7fd7697fa640 20 client.5158.objecter put_session s=0x7fd744080ca0 osd=9 4 2026-02-22T23:28:38.743 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.741+0000 7fd7697fa640 15 client.5158.objecter _session_command_op_remove 9 1 2026-02-22T23:28:38.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 10 client.5158.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:38.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 15 client.5158.objecter _session_command_op_assign -1 2 2026-02-22T23:28:38.748 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 20 client.5158.objecter _get_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 20 client.5158.objecter put_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 20 client.5158.objecter _get_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 15 client.5158.objecter _session_command_op_remove -1 2 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 20 client.5158.objecter get_session s=0x7fd744080ca0 osd=9 3 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 15 client.5158.objecter _session_command_op_assign 9 2 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 20 client.5158.objecter put_session s=0x7fd744080ca0 osd=9 4 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 10 client.5158.objecter _send_command 2 2026-02-22T23:28:38.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.747+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 --> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd7741574e0 con 0x7fd744081370 2026-02-22T23:28:38.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.753+0000 7fd7697fa640 1 -- 10.20.193.6:0/917934783 <== osd.9 v2:10.20.193.40:6808/979607499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd7741574e0 con 0x7fd744081370 2026-02-22T23:28:38.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.753+0000 7fd7697fa640 10 client.5158.objecter ms_dispatch2 0x7fd774000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:38.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.753+0000 7fd7697fa640 10 client.5158.objecter _finish_command 2 = system:0 2026-02-22T23:28:38.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.753+0000 7fd7697fa640 20 client.5158.objecter put_session s=0x7fd744080ca0 osd=9 4 2026-02-22T23:28:38.755 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.753+0000 7fd7697fa640 15 client.5158.objecter _session_command_op_remove 9 2 2026-02-22T23:28:38.756 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:38.756 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:38.756 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:38.757 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 10 librados: watch_flush enter 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 10 librados: watch_flush exit 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 10 client.5158.objecter close_session for osd.9 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7fd744081370 msgr2=0x7fd744083790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:6808/979607499,v1:10.20.193.40:6809/979607499] conn(0x7fd744081370 0x7fd744083790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 20 client.5158.objecter put_session s=0x7fd744080ca0 osd=9 1 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 20 client.5158.objecter shutdown clearing up homeless session... 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 10 client.5158.objecter successfully canceled tick 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd744078e30 msgr2=0x7fd74407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd744078e30 0x7fd74407b2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd764002770 tx=0x7fd764009040 comp rx=0 tx=0).stop 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 10 monclient: shutdown 2026-02-22T23:28:38.758 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd7741503e0 msgr2=0x7fd77414b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 --2- 10.20.193.6:0/917934783 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd7741503e0 0x7fd77414b840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd770007d90 tx=0x7fd77000e4b0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd7794c9640 1 -- 10.20.193.6:0/917934783 reap_dead start 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 shutdown_connections 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 >> 10.20.193.6:0/917934783 conn(0x7fd7740cd600 msgr2=0x7fd7740cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 shutdown_connections 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 -- 10.20.193.6:0/917934783 wait complete. 2026-02-22T23:28:38.759 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.754+0000 7fd77af53640 1 librados: shutdown 2026-02-22T23:28:38.763 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2026-02-22T23:28:38.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.832+0000 7efc37016640 10 monclient: get_monmap_and_config 2026-02-22T23:28:38.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.832+0000 7efc37016640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.832+0000 7efc37016640 10 monclient: monmap: 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:38.832891+0000 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:38.832891+0000 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:38.834 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:38.835 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:38.835 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.832+0000 7efc37016640 1 Processor -- start 2026-02-22T23:28:38.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.832+0000 7efc37016640 1 -- start start 2026-02-22T23:28:38.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.832+0000 7efc37016640 10 monclient: init 2026-02-22T23:28:38.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient: _add_conns ranks=[5,1,2,4,0,3] 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7efc3006ff20 0x7efc30070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient(hunting): picked mon.noname-c con 0x7efc3006ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efc3015c970 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7efc30155eb0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc300725b0 con 0x7efc3015c970 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc30072890 con 0x7efc30155eb0 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7efc30072b70 con 0x7efc3006ff20 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.833+0000 7efc37016640 10 monclient(hunting): authenticate will time out at 1393.560518s 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7efc3006ff20 0x7efc30070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc27fff640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7efc3015c970 0x7efc301557a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38312/0 (socket says 10.20.193.6:38312) 2026-02-22T23:28:38.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc27fff640 1 -- 10.20.193.6:0/4185498892 learned_addr learned my addr 10.20.193.6:0/4185498892 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 10 monclient(hunting): get_auth_request con 0x7efc3006ff20 auth_method 0 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.834+0000 7efc34d8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.835+0000 7efc277fe640 1 -- 10.20.193.6:0/4185498892 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613294489 0 0) 0x7efc300725b0 con 0x7efc3015c970 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.835+0000 7efc277fe640 10 monclient(hunting): _negotiate 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.835+0000 7efc277fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.835+0000 7efc277fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.839 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.835+0000 7efc277fe640 10 monclient(hunting): my global_id is 5161 2026-02-22T23:28:38.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.835+0000 7efc277fe640 1 -- 10.20.193.6:0/4185498892 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc1c003620 con 0x7efc3015c970 2026-02-22T23:28:38.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 10 monclient(hunting): handle_auth_done global_id 5309 payload 274 2026-02-22T23:28:38.840 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 1 -- 10.20.193.6:0/4185498892 >> v1:10.20.193.80:6789/0 conn(0x7efc30155eb0 legacy=0x7efc300c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 1 -- 10.20.193.6:0/4185498892 >> v1:10.20.193.40:6789/0 conn(0x7efc3015c970 legacy=0x7efc301557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 1 monclient: found mon.noname-c 2026-02-22T23:28:38.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.836+0000 7efc34d8b640 1 -- 10.20.193.6:0/4185498892 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc30072e50 con 0x7efc3006ff20 2026-02-22T23:28:38.841 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.838+0000 7efc34d8b640 1 --2- 10.20.193.6:0/4185498892 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc3006ff20 0x7efc30070300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc200098a0 tx=0x7efc2002ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e5c4e3be2e0a98 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 1 -- 10.20.193.6:0/4185498892 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2003c070 con 0x7efc3006ff20 2026-02-22T23:28:38.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:38.846 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: dump: 2026-02-22T23:28:38.846 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.847 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 1 -- 10.20.193.6:0/4185498892 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2002fb70 con 0x7efc3006ff20 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 1 -- 10.20.193.6:0/4185498892 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20037440 con 0x7efc3006ff20 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc277fe640 10 monclient: dump: 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.848 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc37016640 5 monclient: authenticate success, global_id 5309 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc37016640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.839+0000 7efc37016640 4 set_mon_vals no callback set 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 10 monclient: shutdown 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 1 -- 10.20.193.6:0/4185498892 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc3006ff20 msgr2=0x7efc30070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 1 --2- 10.20.193.6:0/4185498892 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc3006ff20 0x7efc30070300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc200098a0 tx=0x7efc2002ee60 comp rx=0 tx=0).stop 2026-02-22T23:28:38.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 1 -- 10.20.193.6:0/4185498892 shutdown_connections 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 1 --2- 10.20.193.6:0/4185498892 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc3006ff20 0x7efc30070300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 1 -- 10.20.193.6:0/4185498892 >> 10.20.193.6:0/4185498892 conn(0x7efc300cd780 msgr2=0x7efc300cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 1 -- 10.20.193.6:0/4185498892 shutdown_connections 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.840+0000 7efc37016640 1 -- 10.20.193.6:0/4185498892 wait complete. 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 10 monclient: build_initial_monmap 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 10 monclient: monmap: 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.850 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 1 librados: starting msgr at 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 1 librados: starting objecter 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 1 Processor -- start 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 1 -- start start 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 1 librados: setting wanted keys 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 1 librados: calling monclient init 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.841+0000 7efc37016640 10 monclient: init 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:28:38.851 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efc3006ff20 0x7efc3015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): picked mon.noname-b con 0x7efc3006ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc30155eb0 0x7efc300c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): picked mon.noname-c con 0x7efc30155eb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efc3015c970 0x7efc301f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): picked mon.noname-a con 0x7efc3015c970 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7efc3015d340 con 0x7efc3015c970 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7efc3015d1c0 con 0x7efc3006ff20 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc3015d4c0 con 0x7efc30155eb0 2026-02-22T23:28:38.852 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc37016640 10 monclient(hunting): authenticate will time out at 1393.569054s 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc30155eb0 0x7efc300c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efc3015c970 0x7efc301f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc34d8b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efc3006ff20 0x7efc3015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc30155eb0 0x7efc300c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43096/0 (socket says 10.20.193.6:43096) 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 1 -- 10.20.193.6:0/1700317431 learned_addr learned my addr 10.20.193.6:0/1700317431 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 10 monclient(hunting): get_auth_request con 0x7efc30155eb0 auth_method 0 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 10 monclient(hunting): get_auth_request con 0x7efc3015c970 auth_method 0 2026-02-22T23:28:38.853 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc34d8b640 10 monclient(hunting): get_auth_request con 0x7efc3006ff20 auth_method 0 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc34d8b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc34d8b640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc34d8b640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc27fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc3558c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.854 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc34d8b640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.842+0000 7efc34d8b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.843+0000 7efc34d8b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 10 monclient(hunting): handle_auth_done global_id 5315 payload 824 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 1 -- 10.20.193.6:0/1700317431 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efc3006ff20 msgr2=0x7efc3015ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efc3006ff20 0x7efc3015ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 1 -- 10.20.193.6:0/1700317431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efc3015c970 msgr2=0x7efc301f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efc3015c970 0x7efc301f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 1 monclient: found mon.noname-c 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.844+0000 7efc27fff640 1 -- 10.20.193.6:0/1700317431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc301f3f30 con 0x7efc30155eb0 2026-02-22T23:28:38.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.845+0000 7efc27fff640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc30155eb0 0x7efc300c49e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc1800c500 tx=0x7efc1800c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.845+0000 7efc257fa640 10 client.?.objecter ms_handle_connect 0x7efc30155eb0 2026-02-22T23:28:38.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.845+0000 7efc257fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 1 -- 10.20.193.6:0/1700317431 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18017070 con 0x7efc30155eb0 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: dump: 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.858 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: _finish_auth 0 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: _check_auth_tickets 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 1 -- 10.20.193.6:0/1700317431 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc18005de0 con 0x7efc30155eb0 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 1 -- 10.20.193.6:0/1700317431 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180072b0 con 0x7efc30155eb0 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 5 monclient: authenticate success, global_id 5315 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 10 monclient: _renew_subs 2026-02-22T23:28:38.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc26ffd640 4 set_mon_vals no callback set 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc301f4210 con 0x7efc30155eb0 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc257fa640 10 monclient: dump: 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:38.860 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 10 client.5315.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 10 monclient: _renew_subs 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301f47d0 con 0x7efc30155eb0 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.846+0000 7efc37016640 1 librados: init done 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.847+0000 7efc37016640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.847+0000 7efc37016640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.847+0000 7efc37016640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:38.861 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.847+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30070300 con 0x7efc30155eb0 2026-02-22T23:28:38.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.858+0000 7efc257fa640 1 -- 10.20.193.6:0/1700317431 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efc18007450 con 0x7efc30155eb0 2026-02-22T23:28:38.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.859+0000 7efc257fa640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efc00078e80 0x7efc0007b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:38.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.859+0000 7efc34d8b640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efc00078e80 0x7efc0007b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:38.862 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.859+0000 7efc34d8b640 10 monclient: get_auth_request con 0x7efc00078e80 auth_method 0 2026-02-22T23:28:38.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.860+0000 7efc34d8b640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efc00078e80 0x7efc0007b340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efc20039000 tx=0x7efc20033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:38.863 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.860+0000 7efc257fa640 10 client.5315.objecter ms_handle_connect 0x7efc00078e80 2026-02-22T23:28:38.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.949+0000 7efc257fa640 1 -- 10.20.193.6:0/1700317431 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7efc18007770 con 0x7efc30155eb0 2026-02-22T23:28:38.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.949+0000 7efc257fa640 10 client.5315.objecter ms_dispatch2 0x7efc30000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:38.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.949+0000 7efc257fa640 3 client.5315.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:38.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.949+0000 7efc257fa640 3 client.5315.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:38.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.949+0000 7efc257fa640 20 client.5315.objecter dump_active .. 0 homeless 2026-02-22T23:28:38.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.955+0000 7efc257fa640 1 -- 10.20.193.6:0/1700317431 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7efc18021020 con 0x7efc30155eb0 2026-02-22T23:28:38.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.955+0000 7efc257fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:38.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:38.955+0000 7efc257fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:39.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.037+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "9", "target": ["mon-mgr", ""]}) -- 0x7efc30076f20 con 0x7efc00078e80 2026-02-22T23:28:39.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc257fa640 1 -- 10.20.193.6:0/1700317431 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 9 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efc30076f20 con 0x7efc00078e80 2026-02-22T23:28:39.039 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 9 to deep-scrub 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 10 librados: watch_flush enter 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 10 librados: watch_flush exit 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 20 client.5315.objecter shutdown clearing up homeless session... 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 10 client.5315.objecter successfully canceled tick 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efc00078e80 msgr2=0x7efc0007b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efc00078e80 0x7efc0007b340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efc20039000 tx=0x7efc20033000 comp rx=0 tx=0).stop 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 10 monclient: shutdown 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc30155eb0 msgr2=0x7efc300c49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc30155eb0 0x7efc300c49e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efc1800c500 tx=0x7efc1800c9d0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.041 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 shutdown_connections 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efc00078e80 0x7efc0007b340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efc3015c970 0x7efc301f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efc30155eb0 0x7efc300c49e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 --2- 10.20.193.6:0/1700317431 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efc3006ff20 0x7efc3015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 >> 10.20.193.6:0/1700317431 conn(0x7efc300cd780 msgr2=0x7efc3015d740 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 shutdown_connections 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 -- 10.20.193.6:0/1700317431 wait complete. 2026-02-22T23:28:39.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.038+0000 7efc37016640 1 librados: shutdown 2026-02-22T23:28:39.048 INFO:tasks.ceph:Scrubbing osd.13 2026-02-22T23:28:39.048 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:39.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.116+0000 7fb5f19a0640 10 monclient: get_monmap_and_config 2026-02-22T23:28:39.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.116+0000 7fb5f19a0640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.117 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.116+0000 7fb5f19a0640 10 monclient: monmap: 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:39.116929+0000 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:39.116929+0000 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:39.118 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:39.119 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:39.119 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.116+0000 7fb5f19a0640 1 Processor -- start 2026-02-22T23:28:39.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.116+0000 7fb5f19a0640 1 -- start start 2026-02-22T23:28:39.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.116+0000 7fb5f19a0640 10 monclient: init 2026-02-22T23:28:39.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 10 monclient: _add_conns ranks=[2,1,0,5,3,4] 2026-02-22T23:28:39.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb5ec1503e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:39.121 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb5ec06fdb0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb5ec0c8aa0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ec15a570 con 0x7fb5ec0c8aa0 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ec15a850 con 0x7fb5ec06fdb0 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ec0c7fc0 con 0x7fb5ec1503e0 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5f19a0640 10 monclient(hunting): authenticate will time out at 1393.844453s 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5eb7fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fb5ec0c8aa0 0x7fb5ec0c8f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52216/0 (socket says 10.20.193.6:52216) 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.117+0000 7fb5eb7fe640 1 -- 10.20.193.6:0/2534522919 learned_addr learned my addr 10.20.193.6:0/2534522919 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086572637 0 0) 0x7fb5ec0c7fc0 con 0x7fb5ec1503e0 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.122 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): my global_id is 5318 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d8003620 con 0x7fb5ec1503e0 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536459886 0 0) 0x7fb5ec15a570 con 0x7fb5ec0c8aa0 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): my global_id is 5211 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5ec0c7fc0 con 0x7fb5ec0c8aa0 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2193379756 0 0) 0x7fb5ec15a850 con 0x7fb5ec06fdb0 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.123 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 10 monclient(hunting): my global_id is 5167 2026-02-22T23:28:39.124 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.119+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5ec15a570 con 0x7fb5ec06fdb0 2026-02-22T23:28:39.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2353549284 0 0) 0x7fb5ec0c7fc0 con 0x7fb5ec0c8aa0 2026-02-22T23:28:39.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 >> v1:10.20.193.80:6789/0 conn(0x7fb5ec1503e0 legacy=0x7fb5ec159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 >> v1:10.20.193.40:6789/0 conn(0x7fb5ec06fdb0 legacy=0x7fb5ec0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec0c82a0 con 0x7fb5ec0c8aa0 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fb5dc003260 con 0x7fb5ec0c8aa0 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5f19a0640 5 monclient: authenticate success, global_id 5211 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5f19a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: dump: 2026-02-22T23:28:39.135 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 >> v1:10.20.193.6:6789/0 conn(0x7fb5ec0c8aa0 legacy=0x7fb5ec0c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5d8003fe0 0x7fb5d8004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): picked mon.a con 0x7fb5d8003fe0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8004e40 0x7fb5d80052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): picked mon.b con 0x7fb5d8004e40 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5d8009ab0 0x7fb5d800bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): picked mon.c con 0x7fb5d8009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d800c490 con 0x7fb5d8003fe0 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d800c640 con 0x7fb5d8004e40 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d800c840 con 0x7fb5d8009ab0 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5e9ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5f19a0640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.121+0000 7fb5eb7fe640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5d8009ab0 0x7fb5d800bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eb7fe640 10 monclient(hunting): get_auth_request con 0x7fb5d8009ab0 auth_method 0 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eb7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8004e40 0x7fb5d80052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.138 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5d8003fe0 0x7fb5d8004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 10 monclient(hunting): get_auth_request con 0x7fb5d8004e40 auth_method 0 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 10 monclient(hunting): get_auth_request con 0x7fb5d8003fe0 auth_method 0 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:39.139 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.122+0000 7fb5eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 10 monclient(hunting): handle_auth_done global_id 5211 payload 293 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 1 -- 10.20.193.6:0/2534522919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8004e40 msgr2=0x7fb5d80052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8004e40 0x7fb5d80052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 1 -- 10.20.193.6:0/2534522919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5d8003fe0 msgr2=0x7fb5d8004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5d8003fe0 0x7fb5d8004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 1 monclient: found mon.c 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 1 -- 10.20.193.6:0/2534522919 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec0c7fc0 con 0x7fb5d8009ab0 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.123+0000 7fb5eb7fe640 1 -- 10.20.193.6:0/2534522919 reap_dead start 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.124+0000 7fb5eb7fe640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5d8009ab0 0x7fb5d800bea0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a6f0 tx=0x7fb5dc02faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed0bd14b91ad585 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00a310 con 0x7fb5d8009ab0 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 10 monclient: dump: 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.141 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5f19a0640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5dc0073d0 con 0x7fb5d8009ab0 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 1 -- 10.20.193.6:0/2534522919 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0077c0 con 0x7fb5d8009ab0 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5e9ffb640 10 monclient: dump: 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5f19a0640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:39.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.125+0000 7fb5f19a0640 4 set_mon_vals no callback set 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 10 monclient: shutdown 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2534522919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5d8009ab0 msgr2=0x7fb5d800bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5d8009ab0 0x7fb5d800bea0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00a6f0 tx=0x7fb5dc02faa0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2534522919 shutdown_connections 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 1 --2- 10.20.193.6:0/2534522919 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5d8009ab0 0x7fb5d800bea0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2534522919 >> 10.20.193.6:0/2534522919 conn(0x7fb5ec0cd600 msgr2=0x7fb5ec0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2534522919 shutdown_connections 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2534522919 wait complete. 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.126+0000 7fb5f19a0640 10 monclient: monmap: 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.144 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 librados: starting msgr at 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 librados: starting objecter 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 Processor -- start 2026-02-22T23:28:39.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 -- start start 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 librados: setting wanted keys 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 librados: calling monclient init 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient: init 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8009ab0 0x7fb5ec0c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): picked mon.noname-b con 0x7fb5d8009ab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5ec15b8c0 0x7fb5ec15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): picked mon.noname-c con 0x7fb5ec15b8c0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5ec1f7100 0x7fb5ec152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): picked mon.noname-a con 0x7fb5ec1f7100 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:39.146 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec0c7d70 con 0x7fb5ec1f7100 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1509d0 con 0x7fb5d8009ab0 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec0c8610 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5f19a0640 10 monclient(hunting): authenticate will time out at 1393.854548s 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5eb7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5ec1f7100 0x7fb5ec152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5eaffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8009ab0 0x7fb5ec0c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.127+0000 7fb5ea7fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5ec15b8c0 0x7fb5ec15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5ec1f7100 0x7fb5ec152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:36946/0 (socket says 10.20.193.6:36946) 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 1 -- 10.20.193.6:0/2975950135 learned_addr learned my addr 10.20.193.6:0/2975950135 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:39.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 10 monclient(hunting): get_auth_request con 0x7fb5ec1f7100 auth_method 0 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5ea7fc640 10 monclient(hunting): get_auth_request con 0x7fb5ec15b8c0 auth_method 0 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5ea7fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5ea7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5ea7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eaffd640 10 monclient(hunting): get_auth_request con 0x7fb5d8009ab0 auth_method 0 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eaffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eaffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eaffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5ea7fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5ea7fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5ea7fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eaffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eaffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eaffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.128+0000 7fb5eb7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 10 monclient(hunting): handle_auth_done global_id 5321 payload 824 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 1 -- 10.20.193.6:0/2975950135 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8009ab0 msgr2=0x7fb5ec0c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb5d8009ab0 0x7fb5ec0c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 1 -- 10.20.193.6:0/2975950135 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5ec1f7100 msgr2=0x7fb5ec152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.149 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb5ec1f7100 0x7fb5ec152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 1 monclient: found mon.noname-c 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.129+0000 7fb5ea7fc640 1 -- 10.20.193.6:0/2975950135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec15bdf0 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.130+0000 7fb5ea7fc640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5ec15b8c0 0x7fb5ec15d940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400ead0 tx=0x7fb5d400c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.130+0000 7fb5c3fff640 10 client.?.objecter ms_handle_connect 0x7fb5ec15b8c0 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.130+0000 7fb5c3fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400ca50 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: dump: 2026-02-22T23:28:39.150 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.151 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d401a070 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400ce50 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5c3fff640 10 monclient: dump: 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.152 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5e97fa640 4 set_mon_vals no callback set 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 5 monclient: authenticate success, global_id 5321 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 monclient: _renew_subs 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec15c0d0 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 client.5321.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 monclient: _renew_subs 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec152550 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 1 librados: init done 2026-02-22T23:28:39.153 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 client.5321.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 15 client.5321.objecter _session_command_op_assign -1 1 2026-02-22T23:28:39.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 20 client.5321.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:39.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 client.5321.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.154 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.132+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5ec15c3b0 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.143+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb5d4014070 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.143+0000 7fb5c3fff640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb5bc078fd0 0x7fb5bc07b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.143+0000 7fb5eaffd640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb5bc078fd0 0x7fb5bc07b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.143+0000 7fb5eaffd640 10 monclient: get_auth_request con 0x7fb5bc078fd0 auth_method 0 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 10 client.5321.objecter ms_handle_connect 0x7fb5bc078fd0 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5eaffd640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb5bc078fd0 0x7fb5bc07b490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005470 tx=0x7fb5e0007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fb5d4097ee0 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 10 client.5321.objecter ms_dispatch2 0x7fb5ec000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:39.156 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 3 client.5321.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 3 client.5321.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 10 client.5321.objecter checking command 1 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fb5bc0814b0 0x7fb5bc0838d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 20 client.5321.objecter _get_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 20 client.5321.objecter put_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 15 client.5321.objecter _session_command_op_remove -1 1 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 20 client.5321.objecter _get_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 20 client.5321.objecter get_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 15 client.5321.objecter _session_command_op_assign 13 1 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 20 client.5321.objecter put_session s=0x7fb5bc080de0 osd=13 4 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 10 client.5321.objecter _send_command 1 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 --> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5d4016ec0 con 0x7fb5bc0814b0 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 20 client.5321.objecter dump_active .. 0 homeless 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5d405eab0 con 0x7fb5ec15b8c0 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.144+0000 7fb5c3fff640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:39.157 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.145+0000 7fb5eb7fe640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fb5bc0814b0 0x7fb5bc0838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.145+0000 7fb5eb7fe640 10 monclient: get_auth_request con 0x7fb5bc0814b0 auth_method 0 2026-02-22T23:28:39.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.148+0000 7fb5eb7fe640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fb5bc0814b0 0x7fb5bc0838d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.13 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.158 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.148+0000 7fb5c3fff640 10 client.5321.objecter ms_handle_connect 0x7fb5bc0814b0 2026-02-22T23:28:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.158+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== osd.13 v2:10.20.193.40:6816/1944897191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fb5d4016ec0 con 0x7fb5bc0814b0 2026-02-22T23:28:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.158+0000 7fb5c3fff640 10 client.5321.objecter ms_dispatch2 0x7fb5ec000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.158+0000 7fb5c3fff640 10 client.5321.objecter _finish_command 1 = system:0 2026-02-22T23:28:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.158+0000 7fb5c3fff640 20 client.5321.objecter put_session s=0x7fb5bc080de0 osd=13 4 2026-02-22T23:28:39.159 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.158+0000 7fb5c3fff640 15 client.5321.objecter _session_command_op_remove 13 1 2026-02-22T23:28:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 10 client.5321.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 15 client.5321.objecter _session_command_op_assign -1 2 2026-02-22T23:28:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 20 client.5321.objecter _get_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.165 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 20 client.5321.objecter put_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 20 client.5321.objecter _get_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 15 client.5321.objecter _session_command_op_remove -1 2 2026-02-22T23:28:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 20 client.5321.objecter get_session s=0x7fb5bc080de0 osd=13 3 2026-02-22T23:28:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 15 client.5321.objecter _session_command_op_assign 13 2 2026-02-22T23:28:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 20 client.5321.objecter put_session s=0x7fb5bc080de0 osd=13 4 2026-02-22T23:28:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 10 client.5321.objecter _send_command 2 2026-02-22T23:28:39.166 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.164+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 --> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb5ec0701d0 con 0x7fb5bc0814b0 2026-02-22T23:28:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5c3fff640 1 -- 10.20.193.6:0/2975950135 <== osd.13 v2:10.20.193.40:6816/1944897191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb5ec0701d0 con 0x7fb5bc0814b0 2026-02-22T23:28:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5c3fff640 10 client.5321.objecter ms_dispatch2 0x7fb5ec000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5c3fff640 10 client.5321.objecter _finish_command 2 = system:0 2026-02-22T23:28:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5c3fff640 20 client.5321.objecter put_session s=0x7fb5bc080de0 osd=13 4 2026-02-22T23:28:39.172 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5c3fff640 15 client.5321.objecter _session_command_op_remove 13 2 2026-02-22T23:28:39.173 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:39.173 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:39.173 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 10 librados: watch_flush enter 2026-02-22T23:28:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 10 librados: watch_flush exit 2026-02-22T23:28:39.174 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 10 client.5321.objecter close_session for osd.13 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fb5bc0814b0 msgr2=0x7fb5bc0838d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.40:6816/1944897191,v1:10.20.193.40:6817/1944897191] conn(0x7fb5bc0814b0 0x7fb5bc0838d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 20 client.5321.objecter put_session s=0x7fb5bc080de0 osd=13 1 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 20 client.5321.objecter shutdown clearing up homeless session... 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 10 client.5321.objecter successfully canceled tick 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb5bc078fd0 msgr2=0x7fb5bc07b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb5bc078fd0 0x7fb5bc07b490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0005470 tx=0x7fb5e0007dc0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 10 monclient: shutdown 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5ec15b8c0 msgr2=0x7fb5ec15d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.175 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 --2- 10.20.193.6:0/2975950135 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb5ec15b8c0 0x7fb5ec15d940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400ead0 tx=0x7fb5d400c0e0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5eb7fe640 1 -- 10.20.193.6:0/2975950135 reap_dead start 2026-02-22T23:28:39.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 shutdown_connections 2026-02-22T23:28:39.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 >> 10.20.193.6:0/2975950135 conn(0x7fb5ec0cd600 msgr2=0x7fb5ec06d720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 shutdown_connections 2026-02-22T23:28:39.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 -- 10.20.193.6:0/2975950135 wait complete. 2026-02-22T23:28:39.176 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.171+0000 7fb5f19a0640 1 librados: shutdown 2026-02-22T23:28:39.180 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2026-02-22T23:28:39.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.247+0000 7fa51cecd640 10 monclient: get_monmap_and_config 2026-02-22T23:28:39.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.247+0000 7fa51cecd640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.247+0000 7fa51cecd640 10 monclient: monmap: 2026-02-22T23:28:39.248 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:39.247758+0000 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:39.247758+0000 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:39.249 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.247+0000 7fa51cecd640 1 Processor -- start 2026-02-22T23:28:39.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.247+0000 7fa51cecd640 1 -- start start 2026-02-22T23:28:39.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.247+0000 7fa51cecd640 10 monclient: init 2026-02-22T23:28:39.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient: _add_conns ranks=[1,3,5,0,2,4] 2026-02-22T23:28:39.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa51806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:39.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa5180c99a0 0x7fa518070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient(hunting): picked mon.noname-a con 0x7fa5180c99a0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa51815a5e0 0x7fa51814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient(hunting): picked mon.noname-c con 0x7fa51815a5e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5180c7e90 con 0x7fa51806ff20 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fa518159a40 con 0x7fa5180c99a0 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa5180c8170 con 0x7fa51815a5e0 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa51cecd640 10 monclient(hunting): authenticate will time out at 1393.975212s 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa5180c99a0 0x7fa518070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa516d76640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fa51815a5e0 0x7fa51814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fa5180c99a0 0x7fa518070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:36962/0 (socket says 10.20.193.6:36962) 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 1 -- 10.20.193.6:0/2796046792 learned_addr learned my addr 10.20.193.6:0/2796046792 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 10 monclient(hunting): get_auth_request con 0x7fa5180c99a0 auth_method 0 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa516575640 1 --1- 10.20.193.6:0/2796046792 >> v1:10.20.193.40:6789/0 conn(0x7fa51806ff20 0x7fa518159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38320/0 (socket says 10.20.193.6:38320) 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa516d76640 10 monclient(hunting): get_auth_request con 0x7fa51815a5e0 auth_method 0 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa516d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa516d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa516d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.248+0000 7fa515d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa516d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa516d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa516d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 10 monclient(hunting): handle_auth_done global_id 5217 payload 274 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 1 -- 10.20.193.6:0/2796046792 >> v2:10.20.193.80:3300/0 conn(0x7fa51815a5e0 msgr2=0x7fa51814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 1 --2- 10.20.193.6:0/2796046792 >> v2:10.20.193.80:3300/0 conn(0x7fa51815a5e0 0x7fa51814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 1 -- 10.20.193.6:0/2796046792 >> v1:10.20.193.40:6789/0 conn(0x7fa51806ff20 legacy=0x7fa518159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 1 monclient: found mon.noname-a 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.249+0000 7fa515d74640 1 -- 10.20.193.6:0/2796046792 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5180c8450 con 0x7fa5180c99a0 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.250+0000 7fa515d74640 1 --2- 10.20.193.6:0/2796046792 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5180c99a0 0x7fa518070300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa500009970 tx=0x7fa50002eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd900f62c3e2db server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa515573640 1 -- 10.20.193.6:0/2796046792 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50003c070 con 0x7fa5180c99a0 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa515573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa515573640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa515573640 10 monclient: dump: 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.257 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.258 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.258 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa515573640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa515573640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa515573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa51cecd640 5 monclient: authenticate success, global_id 5217 2026-02-22T23:28:39.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.251+0000 7fa51cecd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa515573640 1 -- 10.20.193.6:0/2796046792 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50002fbc0 con 0x7fa5180c99a0 2026-02-22T23:28:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa515573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa515573640 1 -- 10.20.193.6:0/2796046792 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa500037440 con 0x7fa5180c99a0 2026-02-22T23:28:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa515573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa515573640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:39.264 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa515573640 10 monclient: dump: 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:39.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 4 set_mon_vals no callback set 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 10 monclient: shutdown 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 -- 10.20.193.6:0/2796046792 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5180c99a0 msgr2=0x7fa518070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 --2- 10.20.193.6:0/2796046792 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5180c99a0 0x7fa518070300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa500009970 tx=0x7fa50002eef0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 -- 10.20.193.6:0/2796046792 shutdown_connections 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 --2- 10.20.193.6:0/2796046792 >> v2:10.20.193.80:3300/0 conn(0x7fa51815a5e0 0x7fa51814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 --2- 10.20.193.6:0/2796046792 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa5180c99a0 0x7fa518070300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 -- 10.20.193.6:0/2796046792 >> 10.20.193.6:0/2796046792 conn(0x7fa5180cd780 msgr2=0x7fa5180cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 -- 10.20.193.6:0/2796046792 shutdown_connections 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.252+0000 7fa51cecd640 1 -- 10.20.193.6:0/2796046792 wait complete. 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient: monmap: 2026-02-22T23:28:39.266 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 librados: starting msgr at 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 librados: starting objecter 2026-02-22T23:28:39.267 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 Processor -- start 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 -- start start 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 librados: setting wanted keys 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 librados: calling monclient init 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient: init 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa51806ff20 0x7fa518156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): picked mon.noname-c con 0x7fa51806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa5180c99a0 0x7fa5180c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): picked mon.noname-b con 0x7fa5180c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa51815a5e0 0x7fa5180c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): picked mon.noname-a con 0x7fa51815a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:39.268 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa518150350 con 0x7fa51815a5e0 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa5180c8a10 con 0x7fa5180c99a0 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181504d0 con 0x7fa51806ff20 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.253+0000 7fa51cecd640 10 monclient(hunting): authenticate will time out at 1393.980808s 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa51815a5e0 0x7fa5180c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa51815a5e0 0x7fa5180c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:36970/0 (socket says 10.20.193.6:36970) 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 1 -- 10.20.193.6:0/3115346840 learned_addr learned my addr 10.20.193.6:0/3115346840 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 10 monclient(hunting): get_auth_request con 0x7fa51815a5e0 auth_method 0 2026-02-22T23:28:39.269 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa51806ff20 0x7fa518156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa5180c99a0 0x7fa5180c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516d76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 10 monclient(hunting): get_auth_request con 0x7fa51806ff20 auth_method 0 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.270 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 10 monclient(hunting): get_auth_request con 0x7fa5180c99a0 auth_method 0 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa516575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.254+0000 7fa515d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 10 monclient(hunting): handle_auth_done global_id 5220 payload 824 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 1 -- 10.20.193.6:0/3115346840 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa51806ff20 msgr2=0x7fa518156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa51806ff20 0x7fa518156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 1 -- 10.20.193.6:0/3115346840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa5180c99a0 msgr2=0x7fa5180c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa5180c99a0 0x7fa5180c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 1 monclient: found mon.noname-a 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.255+0000 7fa516d76640 1 -- 10.20.193.6:0/3115346840 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51806e230 con 0x7fa51815a5e0 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.256+0000 7fa516d76640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa51815a5e0 0x7fa5180c29a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa508009d30 tx=0x7fa50800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.256+0000 7fa4feffd640 10 client.?.objecter ms_handle_connect 0x7fa51815a5e0 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.256+0000 7fa4feffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 1 -- 10.20.193.6:0/3115346840 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508018020 con 0x7fa51815a5e0 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: dump: 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.273 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 1 -- 10.20.193.6:0/3115346840 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa508014480 con 0x7fa51815a5e0 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 1 -- 10.20.193.6:0/3115346840 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508014870 con 0x7fa51815a5e0 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa4feffd640 10 monclient: dump: 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.274 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa514d72640 4 set_mon_vals no callback set 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 5 monclient: authenticate success, global_id 5220 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 monclient: _renew_subs 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51806e510 con 0x7fa51815a5e0 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 client.5220.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 monclient: _renew_subs 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51806ead0 con 0x7fa51815a5e0 2026-02-22T23:28:39.275 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 1 librados: init done 2026-02-22T23:28:39.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.276 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.259+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa518150860 con 0x7fa51815a5e0 2026-02-22T23:28:39.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.271+0000 7fa4feffd640 1 -- 10.20.193.6:0/3115346840 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa508014a10 con 0x7fa51815a5e0 2026-02-22T23:28:39.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.271+0000 7fa4feffd640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa4e4078e30 0x7fa4e407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.271+0000 7fa516575640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa4e4078e30 0x7fa4e407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.271+0000 7fa516575640 10 monclient: get_auth_request con 0x7fa4e4078e30 auth_method 0 2026-02-22T23:28:39.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.272+0000 7fa516575640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa4e4078e30 0x7fa4e407b2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa50c002770 tx=0x7fa50c007f20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.277 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.272+0000 7fa4feffd640 10 client.5220.objecter ms_handle_connect 0x7fa4e4078e30 2026-02-22T23:28:39.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 1 -- 10.20.193.6:0/3115346840 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fa50805ed70 con 0x7fa51815a5e0 2026-02-22T23:28:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 10 client.5220.objecter ms_dispatch2 0x7fa518000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 3 client.5220.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 3 client.5220.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 20 client.5220.objecter dump_active .. 0 homeless 2026-02-22T23:28:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 1 -- 10.20.193.6:0/3115346840 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fa508022070 con 0x7fa51815a5e0 2026-02-22T23:28:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.369+0000 7fa4feffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:39.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.453+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "13", "target": ["mon-mgr", ""]}) -- 0x7fa518157f30 con 0x7fa4e4078e30 2026-02-22T23:28:39.455 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa4feffd640 1 -- 10.20.193.6:0/3115346840 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 13 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7fa518157f30 con 0x7fa4e4078e30 2026-02-22T23:28:39.455 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 13 to deep-scrub 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 10 librados: watch_flush enter 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 10 librados: watch_flush exit 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 20 client.5220.objecter shutdown clearing up homeless session... 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 10 client.5220.objecter successfully canceled tick 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa4e4078e30 msgr2=0x7fa4e407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa4e4078e30 0x7fa4e407b2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa50c002770 tx=0x7fa50c007f20 comp rx=0 tx=0).stop 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 10 monclient: shutdown 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa51815a5e0 msgr2=0x7fa5180c29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.457 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.454+0000 7fa51cecd640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa51815a5e0 0x7fa5180c29a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa508009d30 tx=0x7fa50800c490 comp rx=0 tx=0).stop 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 shutdown_connections 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa4e4078e30 0x7fa4e407b2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa51815a5e0 0x7fa5180c29a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa5180c99a0 0x7fa5180c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 --2- 10.20.193.6:0/3115346840 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa51806ff20 0x7fa518156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 >> 10.20.193.6:0/3115346840 conn(0x7fa5180cd780 msgr2=0x7fa51814e690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 shutdown_connections 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 -- 10.20.193.6:0/3115346840 wait complete. 2026-02-22T23:28:39.458 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.455+0000 7fa51cecd640 1 librados: shutdown 2026-02-22T23:28:39.465 INFO:tasks.ceph:Scrubbing osd.2 2026-02-22T23:28:39.465 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:39.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.532+0000 7fd3a4457640 10 monclient: get_monmap_and_config 2026-02-22T23:28:39.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.532+0000 7fd3a4457640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient: monmap: 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:39.533624+0000 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:39.533624+0000 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:39.535 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:39.536 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:39.536 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:39.536 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 1 Processor -- start 2026-02-22T23:28:39.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 1 -- start start 2026-02-22T23:28:39.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient: init 2026-02-22T23:28:39.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient: _add_conns ranks=[3,5,4,2,1,0] 2026-02-22T23:28:39.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd39c1503e0 0x7fd39c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient(hunting): picked mon.noname-a con 0x7fd39c1503e0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:39.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd39c15a070 0x7fd39c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient(hunting): picked mon.noname-c con 0x7fd39c15a070 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:39.544 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fd39c0702f0 0x7fd39c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient(hunting): picked mon.noname-b con 0x7fd39c0702f0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.533+0000 7fd3a4457640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fd39c0c7d70 con 0x7fd39c1503e0 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a4457640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a4457640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fd39c0c8610 con 0x7fd39c0702f0 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a4457640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a4457640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd39c070760 con 0x7fd39c15a070 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a4457640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a4457640 10 monclient(hunting): authenticate will time out at 1394.261005s 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd39c1503e0 0x7fd39c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a29cd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fd39c0702f0 0x7fd39c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a19cb640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd39c15a070 0x7fd39c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fd39c1503e0 0x7fd39c159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:36976/0 (socket says 10.20.193.6:36976) 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 1 -- 10.20.193.6:0/2411829574 learned_addr learned my addr 10.20.193.6:0/2411829574 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 10 monclient(hunting): get_auth_request con 0x7fd39c1503e0 auth_method 0 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a29cd640 10 monclient(hunting): get_auth_request con 0x7fd39c0702f0 auth_method 0 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a29cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a29cd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a29cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a19cb640 10 monclient(hunting): get_auth_request con 0x7fd39c15a070 auth_method 0 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a19cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a19cb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a19cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.534+0000 7fd3a21cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a19cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a19cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a19cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a29cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a29cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a29cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 10 monclient(hunting): handle_auth_done global_id 5223 payload 274 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 1 -- 10.20.193.6:0/2411829574 >> v2:10.20.193.80:3300/0 conn(0x7fd39c15a070 msgr2=0x7fd39c06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 1 --2- 10.20.193.6:0/2411829574 >> v2:10.20.193.80:3300/0 conn(0x7fd39c15a070 0x7fd39c06fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.547 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 1 -- 10.20.193.6:0/2411829574 >> v2:10.20.193.40:3300/0 conn(0x7fd39c0702f0 msgr2=0x7fd39c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 1 --2- 10.20.193.6:0/2411829574 >> v2:10.20.193.40:3300/0 conn(0x7fd39c0702f0 0x7fd39c0c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 1 monclient: found mon.noname-a 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.535+0000 7fd3a21cc640 1 -- 10.20.193.6:0/2411829574 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c15a570 con 0x7fd39c1503e0 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.536+0000 7fd3a21cc640 1 --2- 10.20.193.6:0/2411829574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c1503e0 0x7fd39c159b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0c8790 tx=0x7fd38c02ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e3f7dc3dbef476 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 1 -- 10.20.193.6:0/2411829574 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c03c070 con 0x7fd39c1503e0 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: dump: 2026-02-22T23:28:39.548 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.549 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 1 -- 10.20.193.6:0/2411829574 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38c02fbd0 con 0x7fd39c1503e0 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 5 monclient: authenticate success, global_id 5223 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 1 -- 10.20.193.6:0/2411829574 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c037440 con 0x7fd39c1503e0 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a11ca640 10 monclient: dump: 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.550 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 4 set_mon_vals no callback set 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 10 monclient: shutdown 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 -- 10.20.193.6:0/2411829574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c1503e0 msgr2=0x7fd39c159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 --2- 10.20.193.6:0/2411829574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c1503e0 0x7fd39c159b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0c8790 tx=0x7fd38c02ef10 comp rx=0 tx=0).stop 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 -- 10.20.193.6:0/2411829574 shutdown_connections 2026-02-22T23:28:39.551 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 --2- 10.20.193.6:0/2411829574 >> v2:10.20.193.40:3300/0 conn(0x7fd39c0702f0 0x7fd39c0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 --2- 10.20.193.6:0/2411829574 >> v2:10.20.193.80:3300/0 conn(0x7fd39c15a070 0x7fd39c06fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 --2- 10.20.193.6:0/2411829574 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c1503e0 0x7fd39c159b30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 -- 10.20.193.6:0/2411829574 >> 10.20.193.6:0/2411829574 conn(0x7fd39c0cd600 msgr2=0x7fd39c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 -- 10.20.193.6:0/2411829574 shutdown_connections 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.538+0000 7fd3a4457640 1 -- 10.20.193.6:0/2411829574 wait complete. 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 10 monclient: monmap: 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.552 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 1 librados: starting msgr at 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 1 librados: starting objecter 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 1 Processor -- start 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 1 -- start start 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 1 librados: setting wanted keys 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 1 librados: calling monclient init 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.539+0000 7fd3a4457640 10 monclient: init 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.553 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient: _add_conns ranks=[1,2,0] 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd39c0702f0 0x7fd39c1f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): picked mon.noname-b con 0x7fd39c0702f0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd39c1503e0 0x7fd39c147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): picked mon.noname-c con 0x7fd39c1503e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c15a070 0x7fd39c0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): picked mon.noname-a con 0x7fd39c15a070 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c0c82a0 con 0x7fd39c15a070 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c0c7fc0 con 0x7fd39c0702f0 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c1507c0 con 0x7fd39c1503e0 2026-02-22T23:28:39.554 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a4457640 10 monclient(hunting): authenticate will time out at 1394.267113s 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c15a070 0x7fd39c0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a19cb640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd39c1503e0 0x7fd39c147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a21cc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd39c0702f0 0x7fd39c1f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c15a070 0x7fd39c0cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:36986/0 (socket says 10.20.193.6:36986) 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 1 -- 10.20.193.6:0/3844441622 learned_addr learned my addr 10.20.193.6:0/3844441622 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 10 monclient(hunting): get_auth_request con 0x7fd39c15a070 auth_method 0 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a19cb640 10 monclient(hunting): get_auth_request con 0x7fd39c1503e0 auth_method 0 2026-02-22T23:28:39.555 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a19cb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a19cb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a19cb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a21cc640 10 monclient(hunting): get_auth_request con 0x7fd39c0702f0 auth_method 0 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a21cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a21cc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a21cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a29cd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a19cb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.540+0000 7fd3a19cb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.541+0000 7fd3a19cb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.556 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.541+0000 7fd3a21cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.541+0000 7fd3a21cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.541+0000 7fd3a21cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 10 monclient(hunting): handle_auth_done global_id 5226 payload 824 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 1 -- 10.20.193.6:0/3844441622 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd39c1503e0 msgr2=0x7fd39c147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd39c1503e0 0x7fd39c147600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 1 -- 10.20.193.6:0/3844441622 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd39c0702f0 msgr2=0x7fd39c1f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd39c0702f0 0x7fd39c1f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 1 monclient: found mon.noname-a 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.557 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.542+0000 7fd3a29cd640 1 -- 10.20.193.6:0/3844441622 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c15d480 con 0x7fd39c15a070 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.543+0000 7fd3a29cd640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c15a070 0x7fd39c0cc220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd39800eed0 tx=0x7fd39800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.543+0000 7fd382ffd640 10 client.?.objecter ms_handle_connect 0x7fd39c15a070 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.543+0000 7fd382ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd398018020 con 0x7fd39c15a070 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: dump: 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.560 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd398014400 con 0x7fd39c15a070 2026-02-22T23:28:39.561 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 5 monclient: authenticate success, global_id 5226 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 monclient: _renew_subs 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c15d760 con 0x7fd39c15a070 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3980147f0 con 0x7fd39c15a070 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd382ffd640 10 monclient: dump: 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.562 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a09c9640 4 set_mon_vals no callback set 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 client.5226.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 monclient: _renew_subs 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c1f3720 con 0x7fd39c15a070 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 1 librados: init done 2026-02-22T23:28:39.563 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 client.5226.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 15 client.5226.objecter _session_command_op_assign -1 1 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 20 client.5226.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 client.5226.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.545+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd39c159b30 con 0x7fd39c15a070 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.557+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd398014990 con 0x7fd39c15a070 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.557+0000 7fd382ffd640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd36c078e00 0x7fd36c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.557+0000 7fd3a21cc640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd36c078e00 0x7fd36c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.557+0000 7fd3a21cc640 10 monclient: get_auth_request con 0x7fd36c078e00 auth_method 0 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.558+0000 7fd382ffd640 10 client.5226.objecter ms_handle_connect 0x7fd36c078e00 2026-02-22T23:28:39.564 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.558+0000 7fd3a21cc640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd36c078e00 0x7fd36c07b2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0ce490 tx=0x7fd38c039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fd398098240 con 0x7fd39c15a070 2026-02-22T23:28:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 10 client.5226.objecter ms_dispatch2 0x7fd39c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 3 client.5226.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 3 client.5226.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 10 client.5226.objecter checking command 1 2026-02-22T23:28:39.566 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd36c081340 0x7fd36c083760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 20 client.5226.objecter _get_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 20 client.5226.objecter put_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 15 client.5226.objecter _session_command_op_remove -1 1 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 20 client.5226.objecter _get_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 20 client.5226.objecter get_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 15 client.5226.objecter _session_command_op_assign 2 1 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 20 client.5226.objecter put_session s=0x7fd36c080c70 osd=2 4 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 10 client.5226.objecter _send_command 1 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 --> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd398014e50 con 0x7fd36c081340 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 20 client.5226.objecter dump_active .. 0 homeless 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd39805ed00 con 0x7fd39c15a070 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd382ffd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:39.567 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.559+0000 7fd3a19cb640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd36c081340 0x7fd36c083760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.560+0000 7fd3a19cb640 10 monclient: get_auth_request con 0x7fd36c081340 auth_method 0 2026-02-22T23:28:39.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.561+0000 7fd3a19cb640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd36c081340 0x7fd36c083760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.568 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.561+0000 7fd382ffd640 10 client.5226.objecter ms_handle_connect 0x7fd36c081340 2026-02-22T23:28:39.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.571+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== osd.2 v2:10.20.193.80:6816/1223585399 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd398014e50 con 0x7fd36c081340 2026-02-22T23:28:39.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.571+0000 7fd382ffd640 10 client.5226.objecter ms_dispatch2 0x7fd39c000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:39.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.571+0000 7fd382ffd640 10 client.5226.objecter _finish_command 1 = system:0 2026-02-22T23:28:39.572 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.571+0000 7fd382ffd640 20 client.5226.objecter put_session s=0x7fd36c080c70 osd=2 4 2026-02-22T23:28:39.573 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.571+0000 7fd382ffd640 15 client.5226.objecter _session_command_op_remove 2 1 2026-02-22T23:28:39.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 10 client.5226.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:39.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 15 client.5226.objecter _session_command_op_assign -1 2 2026-02-22T23:28:39.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 20 client.5226.objecter _get_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 20 client.5226.objecter put_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 20 client.5226.objecter _get_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 15 client.5226.objecter _session_command_op_remove -1 2 2026-02-22T23:28:39.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 20 client.5226.objecter get_session s=0x7fd36c080c70 osd=2 3 2026-02-22T23:28:39.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 15 client.5226.objecter _session_command_op_assign 2 2 2026-02-22T23:28:39.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 20 client.5226.objecter put_session s=0x7fd36c080c70 osd=2 4 2026-02-22T23:28:39.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 10 client.5226.objecter _send_command 2 2026-02-22T23:28:39.579 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.577+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 --> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd39c070d60 con 0x7fd36c081340 2026-02-22T23:28:39.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.583+0000 7fd382ffd640 1 -- 10.20.193.6:0/3844441622 <== osd.2 v2:10.20.193.80:6816/1223585399 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd39c070d60 con 0x7fd36c081340 2026-02-22T23:28:39.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.583+0000 7fd382ffd640 10 client.5226.objecter ms_dispatch2 0x7fd39c000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:39.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.583+0000 7fd382ffd640 10 client.5226.objecter _finish_command 2 = system:0 2026-02-22T23:28:39.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.583+0000 7fd382ffd640 20 client.5226.objecter put_session s=0x7fd36c080c70 osd=2 4 2026-02-22T23:28:39.585 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.583+0000 7fd382ffd640 15 client.5226.objecter _session_command_op_remove 2 2 2026-02-22T23:28:39.586 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:39.586 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:39.586 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:39.587 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 10 librados: watch_flush enter 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 10 librados: watch_flush exit 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 10 client.5226.objecter close_session for osd.2 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd36c081340 msgr2=0x7fd36c083760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.80:6816/1223585399,v1:10.20.193.80:6817/1223585399] conn(0x7fd36c081340 0x7fd36c083760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 20 client.5226.objecter put_session s=0x7fd36c080c70 osd=2 1 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 20 client.5226.objecter shutdown clearing up homeless session... 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 10 client.5226.objecter successfully canceled tick 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd36c078e00 msgr2=0x7fd36c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd36c078e00 0x7fd36c07b2c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0ce490 tx=0x7fd38c039040 comp rx=0 tx=0).stop 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 10 monclient: shutdown 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.588 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c15a070 msgr2=0x7fd39c0cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 --2- 10.20.193.6:0/3844441622 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd39c15a070 0x7fd39c0cc220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd39800eed0 tx=0x7fd39800b480 comp rx=0 tx=0).stop 2026-02-22T23:28:39.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a29cd640 1 -- 10.20.193.6:0/3844441622 reap_dead start 2026-02-22T23:28:39.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 shutdown_connections 2026-02-22T23:28:39.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 >> 10.20.193.6:0/3844441622 conn(0x7fd39c0cd600 msgr2=0x7fd39c14c560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 shutdown_connections 2026-02-22T23:28:39.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 -- 10.20.193.6:0/3844441622 wait complete. 2026-02-22T23:28:39.589 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.584+0000 7fd3a4457640 1 librados: shutdown 2026-02-22T23:28:39.593 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-22T23:28:39.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.660+0000 7f82e858f640 10 monclient: get_monmap_and_config 2026-02-22T23:28:39.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.660+0000 7f82e858f640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.660+0000 7f82e858f640 10 monclient: monmap: 2026-02-22T23:28:39.662 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.662 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.662 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:39.661511+0000 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:39.661511+0000 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:39.663 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 1 Processor -- start 2026-02-22T23:28:39.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 1 -- start start 2026-02-22T23:28:39.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient: init 2026-02-22T23:28:39.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.666 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient: _add_conns ranks=[3,2,0,1,5,4] 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f82e006ff20 0x7f82e0070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient(hunting): picked mon.noname-a con 0x7f82e006ff20 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f82e00c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f82e015a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e00c7e90 con 0x7f82e015a5e0 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e00c8170 con 0x7f82e00c99a0 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f82e0159a40 con 0x7f82e006ff20 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.661+0000 7f82e858f640 10 monclient(hunting): authenticate will time out at 1394.388887s 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f82e006ff20 0x7f82e0070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.667 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e5b03640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f82e00c99a0 0x7f82e0159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:46680/0 (socket says 10.20.193.6:46680) 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e5b03640 1 -- 10.20.193.6:0/2096829743 learned_addr learned my addr 10.20.193.6:0/2096829743 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 10 monclient(hunting): get_auth_request con 0x7f82e006ff20 auth_method 0 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.662+0000 7f82e6304640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e5302640 1 -- 10.20.193.6:0/2096829743 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277570922 0 0) 0x7f82e00c8170 con 0x7f82e00c99a0 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e5302640 10 monclient(hunting): _negotiate 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e5302640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.668 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e5302640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e5302640 10 monclient(hunting): my global_id is 5336 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e5302640 1 -- 10.20.193.6:0/2096829743 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82c8003620 con 0x7f82e00c99a0 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 10 monclient(hunting): handle_auth_done global_id 5229 payload 274 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 1 -- 10.20.193.6:0/2096829743 >> v1:10.20.193.80:6789/0 conn(0x7f82e00c99a0 legacy=0x7f82e0159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 1 -- 10.20.193.6:0/2096829743 >> v1:10.20.193.6:6789/0 conn(0x7f82e015a5e0 legacy=0x7f82e014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 1 monclient: found mon.noname-a 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.663+0000 7f82e6304640 1 -- 10.20.193.6:0/2096829743 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e00c8450 con 0x7f82e006ff20 2026-02-22T23:28:39.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.664+0000 7f82e6304640 1 --2- 10.20.193.6:0/2096829743 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e006ff20 0x7f82e0070300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f82d00098d0 tx=0x7f82d002ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21f87eeffb0522b server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 1 -- 10.20.193.6:0/2096829743 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d003c070 con 0x7f82e006ff20 2026-02-22T23:28:39.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:39.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: dump: 2026-02-22T23:28:39.676 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.676 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 5 monclient: authenticate success, global_id 5229 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 1 -- 10.20.193.6:0/2096829743 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d002fb00 con 0x7f82e006ff20 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 1 -- 10.20.193.6:0/2096829743 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0037440 con 0x7f82e006ff20 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 4 set_mon_vals no callback set 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e5302640 10 monclient: dump: 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.678 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 10 monclient: shutdown 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 1 -- 10.20.193.6:0/2096829743 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e006ff20 msgr2=0x7f82e0070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 1 --2- 10.20.193.6:0/2096829743 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e006ff20 0x7f82e0070300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f82d00098d0 tx=0x7f82d002ee80 comp rx=0 tx=0).stop 2026-02-22T23:28:39.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 1 -- 10.20.193.6:0/2096829743 shutdown_connections 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 1 --2- 10.20.193.6:0/2096829743 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e006ff20 0x7f82e0070300 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 1 -- 10.20.193.6:0/2096829743 >> 10.20.193.6:0/2096829743 conn(0x7f82e00cd780 msgr2=0x7f82e00cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 1 -- 10.20.193.6:0/2096829743 shutdown_connections 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 1 -- 10.20.193.6:0/2096829743 wait complete. 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.667+0000 7f82e858f640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient: monmap: 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.680 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 librados: starting msgr at 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 librados: starting objecter 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 Processor -- start 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 -- start start 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 librados: setting wanted keys 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 librados: calling monclient init 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient: init 2026-02-22T23:28:39.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f82e006ff20 0x7f82e006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): picked mon.noname-b con 0x7f82e006ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e00c99a0 0x7f82e006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): picked mon.noname-a con 0x7f82e00c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82e015a5e0 0x7f82e01f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): picked mon.noname-c con 0x7f82e015a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f82e0150520 con 0x7f82e00c99a0 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f82e00c8a10 con 0x7f82e006ff20 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f82e01506a0 con 0x7f82e015a5e0 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e858f640 10 monclient(hunting): authenticate will time out at 1394.395702s 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.668+0000 7f82e5b03640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e00c99a0 0x7f82e006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e00c99a0 0x7f82e006d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37012/0 (socket says 10.20.193.6:37012) 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 1 -- 10.20.193.6:0/1752398079 learned_addr learned my addr 10.20.193.6:0/1752398079 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 10 monclient(hunting): get_auth_request con 0x7f82e00c99a0 auth_method 0 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82e015a5e0 0x7f82e01f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f82e006ff20 0x7f82e006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e5b03640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 10 monclient(hunting): get_auth_request con 0x7f82e006ff20 auth_method 0 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 10 monclient(hunting): get_auth_request con 0x7f82e015a5e0 auth_method 0 2026-02-22T23:28:39.684 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6b05640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.669+0000 7f82e6304640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 10 monclient(hunting): handle_auth_done global_id 5232 payload 824 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 1 -- 10.20.193.6:0/1752398079 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82e015a5e0 msgr2=0x7f82e01f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82e015a5e0 0x7f82e01f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 1 -- 10.20.193.6:0/1752398079 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f82e006ff20 msgr2=0x7f82e006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f82e006ff20 0x7f82e006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 1 monclient: found mon.noname-a 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.670+0000 7f82e5b03640 1 -- 10.20.193.6:0/1752398079 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e01f3ef0 con 0x7f82e00c99a0 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.671+0000 7f82e5b03640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e00c99a0 0x7f82e006d5a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82d400c730 tx=0x7f82d400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.671+0000 7f82c6ffd640 10 client.?.objecter ms_handle_connect 0x7f82e00c99a0 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.671+0000 7f82c6ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 1 -- 10.20.193.6:0/1752398079 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4017070 con 0x7f82e00c99a0 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:39.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: dump: 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 1 -- 10.20.193.6:0/1752398079 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d4005bb0 con 0x7f82e00c99a0 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 1 -- 10.20.193.6:0/1752398079 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4007070 con 0x7f82e00c99a0 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e4b01640 4 set_mon_vals no callback set 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82c6ffd640 10 monclient: dump: 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.688 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 5 monclient: authenticate success, global_id 5232 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 monclient: _renew_subs 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e006e030 con 0x7f82e00c99a0 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 client.5232.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 monclient: _renew_subs 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.689 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e006e5f0 con 0x7f82e00c99a0 2026-02-22T23:28:39.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 1 librados: init done 2026-02-22T23:28:39.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:39.690 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.673+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e0070300 con 0x7f82e00c99a0 2026-02-22T23:28:39.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.684+0000 7f82c6ffd640 1 -- 10.20.193.6:0/1752398079 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f82d4007210 con 0x7f82e00c99a0 2026-02-22T23:28:39.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.685+0000 7f82c6ffd640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f82b0078e30 0x7f82b007b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.691 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.685+0000 7f82e6304640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f82b0078e30 0x7f82b007b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.685+0000 7f82e6304640 10 monclient: get_auth_request con 0x7f82b0078e30 auth_method 0 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.685+0000 7f82e6304640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f82b0078e30 0x7f82b007b2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f82d0039000 tx=0x7f82d0033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.685+0000 7f82c6ffd640 10 client.5232.objecter ms_handle_connect 0x7f82b0078e30 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.687+0000 7f82c6ffd640 1 -- 10.20.193.6:0/1752398079 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f82d405d4a0 con 0x7f82e00c99a0 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.687+0000 7f82c6ffd640 10 client.5232.objecter ms_dispatch2 0x7f82e0000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.687+0000 7f82c6ffd640 3 client.5232.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.687+0000 7f82c6ffd640 3 client.5232.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:39.692 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.687+0000 7f82c6ffd640 20 client.5232.objecter dump_active .. 0 homeless 2026-02-22T23:28:39.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.784+0000 7f82c6ffd640 1 -- 10.20.193.6:0/1752398079 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f82d4021020 con 0x7f82e00c99a0 2026-02-22T23:28:39.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.784+0000 7f82c6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.785 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.784+0000 7f82c6ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:39.868 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.867+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f82e006dd60 con 0x7f82b0078e30 2026-02-22T23:28:39.869 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.867+0000 7f82c6ffd640 1 -- 10.20.193.6:0/1752398079 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f82e006dd60 con 0x7f82b0078e30 2026-02-22T23:28:39.869 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 2 to deep-scrub 2026-02-22T23:28:39.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 10 librados: watch_flush enter 2026-02-22T23:28:39.870 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 10 librados: watch_flush exit 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 20 client.5232.objecter shutdown clearing up homeless session... 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 10 client.5232.objecter successfully canceled tick 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f82b0078e30 msgr2=0x7f82b007b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f82b0078e30 0x7f82b007b2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f82d0039000 tx=0x7f82d0033000 comp rx=0 tx=0).stop 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 10 monclient: shutdown 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e00c99a0 msgr2=0x7f82e006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e00c99a0 0x7f82e006d5a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f82d400c730 tx=0x7f82d400cc00 comp rx=0 tx=0).stop 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 shutdown_connections 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f82b0078e30 0x7f82b007b2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.871 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f82e015a5e0 0x7f82e01f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f82e00c99a0 0x7f82e006d5a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 --2- 10.20.193.6:0/1752398079 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f82e006ff20 0x7f82e006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 >> 10.20.193.6:0/1752398079 conn(0x7f82e00cd780 msgr2=0x7f82e0072480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 shutdown_connections 2026-02-22T23:28:39.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 -- 10.20.193.6:0/1752398079 wait complete. 2026-02-22T23:28:39.872 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.868+0000 7f82e858f640 1 librados: shutdown 2026-02-22T23:28:39.878 INFO:tasks.ceph:Scrubbing osd.6 2026-02-22T23:28:39.878 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:39.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.947+0000 7fb33bb3e640 10 monclient: get_monmap_and_config 2026-02-22T23:28:39.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.947+0000 7fb33bb3e640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.947+0000 7fb33bb3e640 10 monclient: monmap: 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:39.947913+0000 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:39.947913+0000 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:39.949 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:39.950 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:39.950 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.947+0000 7fb33bb3e640 1 Processor -- start 2026-02-22T23:28:39.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.947+0000 7fb33bb3e640 1 -- start start 2026-02-22T23:28:39.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.947+0000 7fb33bb3e640 10 monclient: init 2026-02-22T23:28:39.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient: _add_conns ranks=[5,2,1,3,4,0] 2026-02-22T23:28:39.952 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fb3341503e0 0x7fb334159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient(hunting): picked mon.noname-c con 0x7fb3341503e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb33415a070 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb3340704c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb33415a570 con 0x7fb3340704c0 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb33415a850 con 0x7fb33415a070 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb3340c7d70 con 0x7fb3341503e0 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33bb3e640 10 monclient(hunting): authenticate will time out at 1394.675354s 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb3398b3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fb3341503e0 0x7fb334159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33a0b4640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fb3340704c0 0x7fb3340c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38334/0 (socket says 10.20.193.6:38334) 2026-02-22T23:28:39.953 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.948+0000 7fb33a0b4640 1 -- 10.20.193.6:0/3308502245 learned_addr learned my addr 10.20.193.6:0/3308502245 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.949+0000 7fb3398b3640 10 monclient(hunting): get_auth_request con 0x7fb3341503e0 auth_method 0 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.949+0000 7fb3398b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.949+0000 7fb3398b3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.949+0000 7fb3398b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.949+0000 7fb3398b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.949+0000 7fb3398b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.949+0000 7fb3398b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3388b1640 1 -- 10.20.193.6:0/3308502245 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923894479 0 0) 0x7fb33415a570 con 0x7fb3340704c0 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3388b1640 10 monclient(hunting): _negotiate 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3388b1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3388b1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3388b1640 10 monclient(hunting): my global_id is 5188 2026-02-22T23:28:39.954 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3388b1640 1 -- 10.20.193.6:0/3308502245 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb31c003620 con 0x7fb3340704c0 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 10 monclient(hunting): handle_auth_done global_id 5342 payload 274 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 1 -- 10.20.193.6:0/3308502245 >> v1:10.20.193.80:6789/0 conn(0x7fb33415a070 legacy=0x7fb33406fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 1 -- 10.20.193.6:0/3308502245 >> v1:10.20.193.40:6789/0 conn(0x7fb3340704c0 legacy=0x7fb3340c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 1 monclient: found mon.noname-c 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.955 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.950+0000 7fb3398b3640 1 -- 10.20.193.6:0/3308502245 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3340c7fc0 con 0x7fb3341503e0 2026-02-22T23:28:39.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.952+0000 7fb3398b3640 1 --2- 10.20.193.6:0/3308502245 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3341503e0 0x7fb334159b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3240098a0 tx=0x7fb32402ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245d7188541c11e9 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 1 -- 10.20.193.6:0/3308502245 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32403c070 con 0x7fb3341503e0 2026-02-22T23:28:39.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:39.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: dump: 2026-02-22T23:28:39.960 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.960 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 1 -- 10.20.193.6:0/3308502245 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb32402fc10 con 0x7fb3341503e0 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb33bb3e640 5 monclient: authenticate success, global_id 5342 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 1 -- 10.20.193.6:0/3308502245 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324037440 con 0x7fb3341503e0 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb33bb3e640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb33bb3e640 4 set_mon_vals no callback set 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.954+0000 7fb3388b1640 10 monclient: dump: 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.962 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 10 monclient: shutdown 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 -- 10.20.193.6:0/3308502245 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3341503e0 msgr2=0x7fb334159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 --2- 10.20.193.6:0/3308502245 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3341503e0 0x7fb334159b30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3240098a0 tx=0x7fb32402ef00 comp rx=0 tx=0).stop 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 -- 10.20.193.6:0/3308502245 shutdown_connections 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 --2- 10.20.193.6:0/3308502245 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3341503e0 0x7fb334159b30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 -- 10.20.193.6:0/3308502245 >> 10.20.193.6:0/3308502245 conn(0x7fb3340cd600 msgr2=0x7fb3340cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 -- 10.20.193.6:0/3308502245 shutdown_connections 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 -- 10.20.193.6:0/3308502245 wait complete. 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 10 monclient: build_initial_monmap 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 10 monclient: monmap: 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:39.964 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 librados: starting msgr at 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 librados: starting objecter 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.955+0000 7fb33bb3e640 1 Processor -- start 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 -- start start 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 librados: setting wanted keys 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 librados: calling monclient init 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient: init 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3340704c0 0x7fb3341592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): picked mon.noname-c con 0x7fb3340704c0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb3341503e0 0x7fb3341f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): picked mon.noname-b con 0x7fb3341503e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb33415a070 0x7fb3341581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): picked mon.noname-a con 0x7fb33415a070 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb3341507c0 con 0x7fb33415a070 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb3340c8610 con 0x7fb3341503e0 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb334150940 con 0x7fb3340704c0 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33bb3e640 10 monclient(hunting): authenticate will time out at 1394.683501s 2026-02-22T23:28:39.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3398b3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3340704c0 0x7fb3341592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3390b2640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb3341503e0 0x7fb3341f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb33a0b4640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb33415a070 0x7fb3341581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3398b3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3340704c0 0x7fb3341592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43172/0 (socket says 10.20.193.6:43172) 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3398b3640 1 -- 10.20.193.6:0/168626979 learned_addr learned my addr 10.20.193.6:0/168626979 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3398b3640 10 monclient(hunting): get_auth_request con 0x7fb3340704c0 auth_method 0 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3398b3640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3398b3640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.956+0000 7fb3398b3640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3390b2640 10 monclient(hunting): get_auth_request con 0x7fb3341503e0 auth_method 0 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3390b2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3390b2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3390b2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb33a0b4640 10 monclient(hunting): get_auth_request con 0x7fb33415a070 auth_method 0 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb33a0b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb33a0b4640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb33a0b4640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3398b3640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3398b3640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3398b3640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3390b2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3390b2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb3390b2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb33a0b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb33a0b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:39.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.957+0000 7fb33a0b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 10 monclient(hunting): handle_auth_done global_id 5348 payload 824 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 1 -- 10.20.193.6:0/168626979 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb3341503e0 msgr2=0x7fb3341f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb3341503e0 0x7fb3341f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 1 -- 10.20.193.6:0/168626979 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb33415a070 msgr2=0x7fb3341581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb33415a070 0x7fb3341581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 10 monclient: _finish_hunting 0 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 1 monclient: found mon.noname-c 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.958+0000 7fb3398b3640 1 -- 10.20.193.6:0/168626979 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33414d8e0 con 0x7fb3340704c0 2026-02-22T23:28:39.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.960+0000 7fb3398b3640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3340704c0 0x7fb3341592d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb324009130 tx=0x7fb3240047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.960+0000 7fb3227fc640 10 client.?.objecter ms_handle_connect 0x7fb3340704c0 2026-02-22T23:28:39.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.960+0000 7fb3227fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:39.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32403c040 con 0x7fb3340704c0 2026-02-22T23:28:39.971 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: dump: 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.972 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb324042630 con 0x7fb3340704c0 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb324042a20 con 0x7fb3340704c0 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb3227fc640 10 monclient: dump: 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.973 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb323fff640 4 set_mon_vals no callback set 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 5 monclient: authenticate success, global_id 5348 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 10 monclient: _renew_subs 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33415cd90 con 0x7fb3340704c0 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 10 client.5348.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.974 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 10 monclient: _renew_subs 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33415d350 con 0x7fb3340704c0 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.962+0000 7fb33bb3e640 1 librados: init done 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.963+0000 7fb33bb3e640 10 client.5348.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.963+0000 7fb33bb3e640 15 client.5348.objecter _session_command_op_assign -1 1 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.963+0000 7fb33bb3e640 20 client.5348.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.963+0000 7fb33bb3e640 10 client.5348.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.963+0000 7fb33bb3e640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:39.975 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.963+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb334159b30 con 0x7fb3340704c0 2026-02-22T23:28:39.976 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.975+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb32404b450 con 0x7fb3340704c0 2026-02-22T23:28:39.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.975+0000 7fb3227fc640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb304078e00 0x7fb30407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3390b2640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb304078e00 0x7fb30407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.977 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3390b2640 10 monclient: get_auth_request con 0x7fb304078e00 auth_method 0 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3390b2640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb304078e00 0x7fb30407b2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb3340c4460 tx=0x7fb328007f30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 10 client.5348.objecter ms_handle_connect 0x7fb304078e00 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fb32402fb50 con 0x7fb3340704c0 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 10 client.5348.objecter ms_dispatch2 0x7fb334000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 3 client.5348.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 3 client.5348.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 10 client.5348.objecter checking command 1 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7fb3040812e0 0x7fb304083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 20 client.5348.objecter _get_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 20 client.5348.objecter put_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.979 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 15 client.5348.objecter _session_command_op_remove -1 1 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 20 client.5348.objecter _get_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 20 client.5348.objecter get_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 15 client.5348.objecter _session_command_op_assign 6 1 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 20 client.5348.objecter put_session s=0x7fb304080c10 osd=6 4 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 10 client.5348.objecter _send_command 1 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 --> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb32404eec0 con 0x7fb3040812e0 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 20 client.5348.objecter dump_active .. 0 homeless 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb324082e20 con 0x7fb3340704c0 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.976+0000 7fb3227fc640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.977+0000 7fb33a0b4640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7fb3040812e0 0x7fb304083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:39.980 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.977+0000 7fb33a0b4640 10 monclient: get_auth_request con 0x7fb3040812e0 auth_method 0 2026-02-22T23:28:39.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.979+0000 7fb33a0b4640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7fb3040812e0 0x7fb304083700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:39.981 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.979+0000 7fb3227fc640 10 client.5348.objecter ms_handle_connect 0x7fb3040812e0 2026-02-22T23:28:39.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.988+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== osd.6 v2:10.20.193.80:6800/144861518 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fb32404eec0 con 0x7fb3040812e0 2026-02-22T23:28:39.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.988+0000 7fb3227fc640 10 client.5348.objecter ms_dispatch2 0x7fb334000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:39.989 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.988+0000 7fb3227fc640 10 client.5348.objecter _finish_command 1 = system:0 2026-02-22T23:28:39.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.988+0000 7fb3227fc640 20 client.5348.objecter put_session s=0x7fb304080c10 osd=6 4 2026-02-22T23:28:39.990 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.988+0000 7fb3227fc640 15 client.5348.objecter _session_command_op_remove 6 1 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 10 client.5348.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 15 client.5348.objecter _session_command_op_assign -1 2 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 20 client.5348.objecter _get_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 20 client.5348.objecter put_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 20 client.5348.objecter _get_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 15 client.5348.objecter _session_command_op_remove -1 2 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 20 client.5348.objecter get_session s=0x7fb304080c10 osd=6 3 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 15 client.5348.objecter _session_command_op_assign 6 2 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 20 client.5348.objecter put_session s=0x7fb304080c10 osd=6 4 2026-02-22T23:28:39.996 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 10 client.5348.objecter _send_command 2 2026-02-22T23:28:39.997 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:39.994+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 --> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb3341598e0 con 0x7fb3040812e0 2026-02-22T23:28:40.009 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb3227fc640 1 -- 10.20.193.6:0/168626979 <== osd.6 v2:10.20.193.80:6800/144861518 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7fb3341598e0 con 0x7fb3040812e0 2026-02-22T23:28:40.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb3227fc640 10 client.5348.objecter ms_dispatch2 0x7fb334000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:40.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb3227fc640 10 client.5348.objecter _finish_command 2 = system:0 2026-02-22T23:28:40.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb3227fc640 20 client.5348.objecter put_session s=0x7fb304080c10 osd=6 4 2026-02-22T23:28:40.010 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb3227fc640 15 client.5348.objecter _session_command_op_remove 6 2 2026-02-22T23:28:40.010 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:40.010 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T23:28:40.011 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 10 librados: watch_flush enter 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 10 librados: watch_flush exit 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 10 client.5348.objecter close_session for osd.6 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7fb3040812e0 msgr2=0x7fb304083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:6800/144861518,v1:10.20.193.80:6801/144861518] conn(0x7fb3040812e0 0x7fb304083700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 20 client.5348.objecter put_session s=0x7fb304080c10 osd=6 1 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 20 client.5348.objecter shutdown clearing up homeless session... 2026-02-22T23:28:40.012 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 10 client.5348.objecter successfully canceled tick 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb304078e00 msgr2=0x7fb30407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb304078e00 0x7fb30407b2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb3340c4460 tx=0x7fb328007f30 comp rx=0 tx=0).stop 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 10 monclient: shutdown 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3340704c0 msgr2=0x7fb3341592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 --2- 10.20.193.6:0/168626979 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb3340704c0 0x7fb3341592d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb324009130 tx=0x7fb3240047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33a0b4640 1 -- 10.20.193.6:0/168626979 reap_dead start 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 shutdown_connections 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 >> 10.20.193.6:0/168626979 conn(0x7fb3340cd600 msgr2=0x7fb3340cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 shutdown_connections 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 -- 10.20.193.6:0/168626979 wait complete. 2026-02-22T23:28:40.013 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.009+0000 7fb33bb3e640 1 librados: shutdown 2026-02-22T23:28:40.017 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2026-02-22T23:28:40.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.081+0000 7ff30f885640 10 monclient: get_monmap_and_config 2026-02-22T23:28:40.083 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.081+0000 7ff30f885640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.081+0000 7ff30f885640 10 monclient: monmap: 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:40.082374+0000 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:40.082374+0000 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:40.084 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:40.085 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:40.085 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 Processor -- start 2026-02-22T23:28:40.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 -- start start 2026-02-22T23:28:40.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient: init 2026-02-22T23:28:40.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient: _add_conns ranks=[5,3,4,0,1,2] 2026-02-22T23:28:40.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff30806ff20 0x7ff308070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): picked mon.noname-c con 0x7ff30806ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:40.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff30815c970 0x7ff30815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): picked mon.noname-a con 0x7ff30815c970 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7ff308073650 0x7ff3081557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): picked mon.noname-b con 0x7ff308073650 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7ff308073a60 con 0x7ff30815c970 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7ff3080725b0 con 0x7ff308073650 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff308072890 con 0x7ff30806ff20 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.082+0000 7ff30f885640 10 monclient(hunting): authenticate will time out at 1394.809809s 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff30806ff20 0x7ff308070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7ff308073650 0x7ff3081557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7ff30815c970 0x7ff30815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7ff30806ff20 0x7ff308070300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43176/0 (socket says 10.20.193.6:43176) 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 1 -- 10.20.193.6:0/1210819138 learned_addr learned my addr 10.20.193.6:0/1210819138 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 10 monclient(hunting): get_auth_request con 0x7ff30806ff20 auth_method 0 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 10 monclient(hunting): get_auth_request con 0x7ff30815c970 auth_method 0 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 10 monclient(hunting): get_auth_request con 0x7ff308073650 auth_method 0 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30d5fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30ddfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.095 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.083+0000 7ff30cdf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 10 monclient(hunting): handle_auth_done global_id 5351 payload 274 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 1 -- 10.20.193.6:0/1210819138 >> v2:10.20.193.40:3300/0 conn(0x7ff308073650 msgr2=0x7ff3081557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 1 --2- 10.20.193.6:0/1210819138 >> v2:10.20.193.40:3300/0 conn(0x7ff308073650 0x7ff3081557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 1 -- 10.20.193.6:0/1210819138 >> v2:10.20.193.6:3300/0 conn(0x7ff30815c970 msgr2=0x7ff30815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 1 --2- 10.20.193.6:0/1210819138 >> v2:10.20.193.6:3300/0 conn(0x7ff30815c970 0x7ff30815d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 1 monclient: found mon.noname-c 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.084+0000 7ff30d5fa640 1 -- 10.20.193.6:0/1210819138 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff308072b70 con 0x7ff30806ff20 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.085+0000 7ff30d5fa640 1 --2- 10.20.193.6:0/1210819138 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 0x7ff308070300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff3080c0e90 tx=0x7ff2f802ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edfb53a4667a352 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 1 -- 10.20.193.6:0/1210819138 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f803c070 con 0x7ff30806ff20 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: dump: 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.097 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 1 -- 10.20.193.6:0/1210819138 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f802fb50 con 0x7ff30806ff20 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 5 monclient: authenticate success, global_id 5351 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 1 -- 10.20.193.6:0/1210819138 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8037440 con 0x7ff30806ff20 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff2f7fff640 10 monclient: dump: 2026-02-22T23:28:40.098 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 4 set_mon_vals no callback set 2026-02-22T23:28:40.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 10 monclient: shutdown 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 -- 10.20.193.6:0/1210819138 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 msgr2=0x7ff308070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 --2- 10.20.193.6:0/1210819138 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 0x7ff308070300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff3080c0e90 tx=0x7ff2f802ee90 comp rx=0 tx=0).stop 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 -- 10.20.193.6:0/1210819138 shutdown_connections 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 --2- 10.20.193.6:0/1210819138 >> v2:10.20.193.40:3300/0 conn(0x7ff308073650 0x7ff3081557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 --2- 10.20.193.6:0/1210819138 >> v2:10.20.193.6:3300/0 conn(0x7ff30815c970 0x7ff30815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 --2- 10.20.193.6:0/1210819138 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 0x7ff308070300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 -- 10.20.193.6:0/1210819138 >> 10.20.193.6:0/1210819138 conn(0x7ff3080cd780 msgr2=0x7ff3080cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 -- 10.20.193.6:0/1210819138 shutdown_connections 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.087+0000 7ff30f885640 1 -- 10.20.193.6:0/1210819138 wait complete. 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient: monmap: 2026-02-22T23:28:40.100 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 librados: starting msgr at 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 librados: starting objecter 2026-02-22T23:28:40.101 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 Processor -- start 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 -- start start 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 librados: setting wanted keys 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 librados: calling monclient init 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient: init 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 0x7ff3081f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): picked mon.noname-c con 0x7ff30806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff308073650 0x7ff3081449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): picked mon.noname-a con 0x7ff308073650 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff30815c970 0x7ff308150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): picked mon.noname-b con 0x7ff30815c970 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7ff308155ce0 con 0x7ff308073650 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7ff308073130 con 0x7ff30815c970 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff308155e60 con 0x7ff30806ff20 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30f885640 10 monclient(hunting): authenticate will time out at 1394.815699s 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.088+0000 7ff30cdf9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff308073650 0x7ff3081449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff30815c970 0x7ff308150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff308073650 0x7ff3081449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37046/0 (socket says 10.20.193.6:37046) 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 1 -- 10.20.193.6:0/3482330363 learned_addr learned my addr 10.20.193.6:0/3482330363 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:40.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 0x7ff3081f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 10 monclient(hunting): get_auth_request con 0x7ff308073650 auth_method 0 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 10 monclient(hunting): get_auth_request con 0x7ff30815c970 auth_method 0 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 10 monclient(hunting): get_auth_request con 0x7ff30806ff20 auth_method 0 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30cdf9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30ddfb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.089+0000 7ff30d5fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 10 monclient(hunting): handle_auth_done global_id 5241 payload 824 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 1 -- 10.20.193.6:0/3482330363 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 msgr2=0x7ff3081f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 0x7ff3081f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 1 -- 10.20.193.6:0/3482330363 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff30815c970 msgr2=0x7ff308150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff30815c970 0x7ff308150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 1 monclient: found mon.noname-a 2026-02-22T23:28:40.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.106 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.090+0000 7ff30cdf9640 1 -- 10.20.193.6:0/3482330363 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3081f4200 con 0x7ff308073650 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.091+0000 7ff30cdf9640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff308073650 0x7ff3081449e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e8c0 tx=0x7ff2fc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.091+0000 7ff2f5ffb640 10 client.?.objecter ms_handle_connect 0x7ff308073650 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.091+0000 7ff2f5ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 1 -- 10.20.193.6:0/3482330363 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc017070 con 0x7ff308073650 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: dump: 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.109 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 1 -- 10.20.193.6:0/3482330363 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2fc00c940 con 0x7ff308073650 2026-02-22T23:28:40.110 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 5 monclient: authenticate success, global_id 5241 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 10 monclient: _renew_subs 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3081f44e0 con 0x7ff308073650 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 1 -- 10.20.193.6:0/3482330363 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00cd30 con 0x7ff308073650 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f5ffb640 10 monclient: dump: 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.111 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff2f77fe640 4 set_mon_vals no callback set 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 10 client.5241.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 10 monclient: _renew_subs 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3080cbaf0 con 0x7ff308073650 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.093+0000 7ff30f885640 1 librados: init done 2026-02-22T23:28:40.112 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.094+0000 7ff30f885640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.094+0000 7ff30f885640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.094+0000 7ff30f885640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.094+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3081557a0 con 0x7ff308073650 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.105+0000 7ff2f5ffb640 1 -- 10.20.193.6:0/3482330363 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7ff2fc013470 con 0x7ff308073650 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.106+0000 7ff2f5ffb640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff2d8078e30 0x7ff2d807b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.106+0000 7ff30d5fa640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff2d8078e30 0x7ff2d807b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.106+0000 7ff30d5fa640 10 monclient: get_auth_request con 0x7ff2d8078e30 auth_method 0 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.106+0000 7ff30d5fa640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff2d8078e30 0x7ff2d807b2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3080c7990 tx=0x7ff2f8039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.106+0000 7ff2f5ffb640 10 client.5241.objecter ms_handle_connect 0x7ff2d8078e30 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.107+0000 7ff2f5ffb640 1 -- 10.20.193.6:0/3482330363 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7ff2fc05e430 con 0x7ff308073650 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.107+0000 7ff2f5ffb640 10 client.5241.objecter ms_dispatch2 0x7ff308000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:40.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.107+0000 7ff2f5ffb640 3 client.5241.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:40.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.107+0000 7ff2f5ffb640 3 client.5241.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:40.114 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.107+0000 7ff2f5ffb640 20 client.5241.objecter dump_active .. 0 homeless 2026-02-22T23:28:40.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.206+0000 7ff2f5ffb640 1 -- 10.20.193.6:0/3482330363 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7ff2fc0623f0 con 0x7ff308073650 2026-02-22T23:28:40.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.206+0000 7ff2f5ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.206+0000 7ff2f5ffb640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:40.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.290+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ff30815c750 con 0x7ff2d8078e30 2026-02-22T23:28:40.292 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.290+0000 7ff2f5ffb640 1 -- 10.20.193.6:0/3482330363 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff30815c750 con 0x7ff2d8078e30 2026-02-22T23:28:40.292 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 6 to deep-scrub 2026-02-22T23:28:40.293 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 10 librados: watch_flush enter 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 10 librados: watch_flush exit 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 20 client.5241.objecter shutdown clearing up homeless session... 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 10 client.5241.objecter successfully canceled tick 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff2d8078e30 msgr2=0x7ff2d807b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff2d8078e30 0x7ff2d807b2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff3080c7990 tx=0x7ff2f8039040 comp rx=0 tx=0).stop 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 10 monclient: shutdown 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff308073650 msgr2=0x7ff3081449e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff308073650 0x7ff3081449e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00e8c0 tx=0x7ff2fc00ed90 comp rx=0 tx=0).stop 2026-02-22T23:28:40.294 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 shutdown_connections 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7ff2d8078e30 0x7ff2d807b2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7ff30815c970 0x7ff308150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7ff308073650 0x7ff3081449e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 --2- 10.20.193.6:0/3482330363 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7ff30806ff20 0x7ff3081f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 >> 10.20.193.6:0/3482330363 conn(0x7ff3080cd780 msgr2=0x7ff30815d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 shutdown_connections 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 -- 10.20.193.6:0/3482330363 wait complete. 2026-02-22T23:28:40.295 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.291+0000 7ff30f885640 1 librados: shutdown 2026-02-22T23:28:40.301 INFO:tasks.ceph:Scrubbing osd.10 2026-02-22T23:28:40.302 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:40.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.370+0000 7efddb985640 10 monclient: get_monmap_and_config 2026-02-22T23:28:40.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.370+0000 7efddb985640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.370+0000 7efddb985640 10 monclient: monmap: 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:40.370881+0000 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:40.370881+0000 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:40.372 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:40.373 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.370+0000 7efddb985640 1 Processor -- start 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.370+0000 7efddb985640 1 -- start start 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.370+0000 7efddb985640 10 monclient: init 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient: _add_conns ranks=[0,1,4,3,5,2] 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7efdd41503e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7efdd406fdb0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7efdd40c8aa0 0x7efdd40c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient(hunting): picked mon.noname-b con 0x7efdd40c8aa0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd40c9450 con 0x7efdd41503e0 2026-02-22T23:28:40.375 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd415a570 con 0x7efdd406fdb0 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7efdd40c7d70 con 0x7efdd40c8aa0 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efddb985640 10 monclient(hunting): authenticate will time out at 1395.098275s 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efdd96fa640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7efdd41503e0 0x7efdd4159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52242/0 (socket says 10.20.193.6:52242) 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efdd96fa640 1 -- 10.20.193.6:0/519126452 learned_addr learned my addr 10.20.193.6:0/519126452 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.371+0000 7efdd9efb640 1 --2- 10.20.193.6:0/519126452 >> v2:10.20.193.40:3300/0 conn(0x7efdd40c8aa0 0x7efdd40c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdd9efb640 10 monclient(hunting): get_auth_request con 0x7efdd40c8aa0 auth_method 0 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdd9efb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdd9efb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdd9efb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdd9efb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdd9efb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdd9efb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdc3fff640 1 -- 10.20.193.6:0/519126452 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375933406 0 0) 0x7efdd40c9450 con 0x7efdd41503e0 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdc3fff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdc3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdc3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdc3fff640 10 monclient(hunting): my global_id is 5244 2026-02-22T23:28:40.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.372+0000 7efdc3fff640 1 -- 10.20.193.6:0/519126452 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdbc003620 con 0x7efdd41503e0 2026-02-22T23:28:40.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 10 monclient(hunting): handle_auth_done global_id 5200 payload 274 2026-02-22T23:28:40.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 1 -- 10.20.193.6:0/519126452 >> v1:10.20.193.40:6789/0 conn(0x7efdd406fdb0 legacy=0x7efdd40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 1 -- 10.20.193.6:0/519126452 >> v1:10.20.193.6:6789/0 conn(0x7efdd41503e0 legacy=0x7efdd4159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 1 monclient: found mon.noname-b 2026-02-22T23:28:40.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.374+0000 7efdd9efb640 1 -- 10.20.193.6:0/519126452 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd415a850 con 0x7efdd40c8aa0 2026-02-22T23:28:40.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.376+0000 7efdd9efb640 1 --2- 10.20.193.6:0/519126452 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 0x7efdd40c8f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009d90 tx=0x7efdd002f2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7332dbac535043a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.378+0000 7efdc3fff640 1 -- 10.20.193.6:0/519126452 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd003c070 con 0x7efdd40c8aa0 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.378+0000 7efdc3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.378+0000 7efdc3fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.378+0000 7efdc3fff640 10 monclient: dump: 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.385 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.378+0000 7efdc3fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.378+0000 7efdc3fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.378+0000 7efdc3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efdc3fff640 1 -- 10.20.193.6:0/519126452 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0037440 con 0x7efdd40c8aa0 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 5 monclient: authenticate success, global_id 5200 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efdc3fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efdc3fff640 1 -- 10.20.193.6:0/519126452 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0037810 con 0x7efdd40c8aa0 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:40.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 4 set_mon_vals no callback set 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efdc3fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efdc3fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efdc3fff640 10 monclient: dump: 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.387 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 10 monclient: shutdown 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 1 -- 10.20.193.6:0/519126452 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 msgr2=0x7efdd40c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 1 --2- 10.20.193.6:0/519126452 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 0x7efdd40c8f10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009d90 tx=0x7efdd002f2c0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 1 -- 10.20.193.6:0/519126452 shutdown_connections 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 1 --2- 10.20.193.6:0/519126452 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 0x7efdd40c8f10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 1 -- 10.20.193.6:0/519126452 >> 10.20.193.6:0/519126452 conn(0x7efdd40cd600 msgr2=0x7efdd40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 1 -- 10.20.193.6:0/519126452 shutdown_connections 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 1 -- 10.20.193.6:0/519126452 wait complete. 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.379+0000 7efddb985640 10 monclient: monmap: 2026-02-22T23:28:40.388 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 librados: starting msgr at 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 librados: starting objecter 2026-02-22T23:28:40.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 Processor -- start 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 -- start start 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 librados: setting wanted keys 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 librados: calling monclient init 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient: init 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efdd406fdb0 0x7efdd414c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): picked mon.noname-c con 0x7efdd406fdb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 0x7efdd4153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): picked mon.noname-b con 0x7efdd40c8aa0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efdd41503e0 0x7efdd414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): picked mon.noname-a con 0x7efdd41503e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:40.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40c82a0 con 0x7efdd41503e0 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40c8610 con 0x7efdd40c8aa0 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7efdd415a240 con 0x7efdd406fdb0 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efddb985640 10 monclient(hunting): authenticate will time out at 1395.107636s 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efdd9efb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efdd41503e0 0x7efdd414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.380+0000 7efdd8ef9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 0x7efdd4153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efdd41503e0 0x7efdd414af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37052/0 (socket says 10.20.193.6:37052) 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 1 -- 10.20.193.6:0/955522731 learned_addr learned my addr 10.20.193.6:0/955522731 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:40.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 10 monclient(hunting): get_auth_request con 0x7efdd41503e0 auth_method 0 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd8ef9640 10 monclient(hunting): get_auth_request con 0x7efdd40c8aa0 auth_method 0 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd8ef9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd8ef9640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd8ef9640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efdd406fdb0 0x7efdd414c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd9efb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd8ef9640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd8ef9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd8ef9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 10 monclient(hunting): get_auth_request con 0x7efdd406fdb0 auth_method 0 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.381+0000 7efdd96fa640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 10 monclient(hunting): handle_auth_done global_id 5247 payload 824 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 1 -- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efdd406fdb0 msgr2=0x7efdd414c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7efdd406fdb0 0x7efdd414c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 1 -- 10.20.193.6:0/955522731 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 msgr2=0x7efdd4153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7efdd40c8aa0 0x7efdd4153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 1 monclient: found mon.noname-a 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.382+0000 7efdd9efb640 1 -- 10.20.193.6:0/955522731 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd414bb60 con 0x7efdd41503e0 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.383+0000 7efdd9efb640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efdd41503e0 0x7efdd414af50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdd003a040 tx=0x7efdd0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.383+0000 7efdc1ffb640 10 client.?.objecter ms_handle_connect 0x7efdd41503e0 2026-02-22T23:28:40.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.383+0000 7efdc1ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:40.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd003c050 con 0x7efdd41503e0 2026-02-22T23:28:40.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:40.396 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: dump: 2026-02-22T23:28:40.396 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.396 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.397 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0048070 con 0x7efdd41503e0 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 5 monclient: authenticate success, global_id 5247 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 monclient: _renew_subs 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd4151ac0 con 0x7efdd41503e0 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0041a30 con 0x7efdd41503e0 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc37fe640 4 set_mon_vals no callback set 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efdc1ffb640 10 monclient: dump: 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.398 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 client.5247.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 monclient: _renew_subs 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd4152060 con 0x7efdd41503e0 2026-02-22T23:28:40.399 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 1 librados: init done 2026-02-22T23:28:40.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 client.5247.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 15 client.5247.objecter _session_command_op_assign -1 1 2026-02-22T23:28:40.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 20 client.5247.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:40.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 client.5247.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:40.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.400 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.385+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdd41507c0 con 0x7efdd41503e0 2026-02-22T23:28:40.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.398+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7efdd0041bd0 con 0x7efdd41503e0 2026-02-22T23:28:40.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.398+0000 7efdc1ffb640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efda4078e80 0x7efda407b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.399+0000 7efdd96fa640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efda4078e80 0x7efda407b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.399+0000 7efdd96fa640 10 monclient: get_auth_request con 0x7efda4078e80 auth_method 0 2026-02-22T23:28:40.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.399+0000 7efdd96fa640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efda4078e80 0x7efda407b340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007b60 tx=0x7efdc4007ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.401 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.399+0000 7efdc1ffb640 10 client.5247.objecter ms_handle_connect 0x7efda4078e80 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7efdd00842b0 con 0x7efdd41503e0 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 10 client.5247.objecter ms_dispatch2 0x7efdd4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 3 client.5247.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 3 client.5247.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 10 client.5247.objecter checking command 1 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7efda4081360 0x7efda4083780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 20 client.5247.objecter _get_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 20 client.5247.objecter put_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 15 client.5247.objecter _session_command_op_remove -1 1 2026-02-22T23:28:40.403 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 20 client.5247.objecter _get_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 20 client.5247.objecter get_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 15 client.5247.objecter _session_command_op_assign 10 1 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 20 client.5247.objecter put_session s=0x7efda4080c90 osd=10 4 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 10 client.5247.objecter _send_command 1 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 --> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdd0037440 con 0x7efda4081360 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 20 client.5247.objecter dump_active .. 0 homeless 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efdd0084670 con 0x7efdd41503e0 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.400+0000 7efdc1ffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.401+0000 7efdd8ef9640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7efda4081360 0x7efda4083780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.404 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.401+0000 7efdd8ef9640 10 monclient: get_auth_request con 0x7efda4081360 auth_method 0 2026-02-22T23:28:40.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.403+0000 7efdd8ef9640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7efda4081360 0x7efda4083780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.405 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.403+0000 7efdc1ffb640 10 client.5247.objecter ms_handle_connect 0x7efda4081360 2026-02-22T23:28:40.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.412+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== osd.10 v2:10.20.193.80:6808/2974128537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7efdd0037440 con 0x7efda4081360 2026-02-22T23:28:40.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.412+0000 7efdc1ffb640 10 client.5247.objecter ms_dispatch2 0x7efdd4000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:40.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.412+0000 7efdc1ffb640 10 client.5247.objecter _finish_command 1 = system:0 2026-02-22T23:28:40.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.413+0000 7efdc1ffb640 20 client.5247.objecter put_session s=0x7efda4080c90 osd=10 4 2026-02-22T23:28:40.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.413+0000 7efdc1ffb640 15 client.5247.objecter _session_command_op_remove 10 1 2026-02-22T23:28:40.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 10 client.5247.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:40.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 15 client.5247.objecter _session_command_op_assign -1 2 2026-02-22T23:28:40.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 20 client.5247.objecter _get_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 20 client.5247.objecter put_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.420 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 20 client.5247.objecter _get_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 15 client.5247.objecter _session_command_op_remove -1 2 2026-02-22T23:28:40.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 20 client.5247.objecter get_session s=0x7efda4080c90 osd=10 3 2026-02-22T23:28:40.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 15 client.5247.objecter _session_command_op_assign 10 2 2026-02-22T23:28:40.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 20 client.5247.objecter put_session s=0x7efda4080c90 osd=10 4 2026-02-22T23:28:40.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 10 client.5247.objecter _send_command 2 2026-02-22T23:28:40.421 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.419+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 --> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efdd406d520 con 0x7efda4081360 2026-02-22T23:28:40.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efdc1ffb640 1 -- 10.20.193.6:0/955522731 <== osd.10 v2:10.20.193.80:6808/2974128537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7efdd406d520 con 0x7efda4081360 2026-02-22T23:28:40.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efdc1ffb640 10 client.5247.objecter ms_dispatch2 0x7efdd4000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:40.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efdc1ffb640 10 client.5247.objecter _finish_command 2 = system:0 2026-02-22T23:28:40.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efdc1ffb640 20 client.5247.objecter put_session s=0x7efda4080c90 osd=10 4 2026-02-22T23:28:40.434 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efdc1ffb640 15 client.5247.objecter _session_command_op_remove 10 2 2026-02-22T23:28:40.435 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:40.435 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T23:28:40.435 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:40.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 10 librados: watch_flush enter 2026-02-22T23:28:40.436 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 10 librados: watch_flush exit 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 10 client.5247.objecter close_session for osd.10 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7efda4081360 msgr2=0x7efda4083780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.80:6808/2974128537,v1:10.20.193.80:6809/2974128537] conn(0x7efda4081360 0x7efda4083780 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 20 client.5247.objecter put_session s=0x7efda4080c90 osd=10 1 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 20 client.5247.objecter shutdown clearing up homeless session... 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 10 client.5247.objecter successfully canceled tick 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efda4078e80 msgr2=0x7efda407b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7efda4078e80 0x7efda407b340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007b60 tx=0x7efdc4007ab0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 10 monclient: shutdown 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.437 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efdd41503e0 msgr2=0x7efdd414af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 --2- 10.20.193.6:0/955522731 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7efdd41503e0 0x7efdd414af50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdd003a040 tx=0x7efdd0004b40 comp rx=0 tx=0).stop 2026-02-22T23:28:40.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efdd9efb640 1 -- 10.20.193.6:0/955522731 reap_dead start 2026-02-22T23:28:40.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 shutdown_connections 2026-02-22T23:28:40.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 >> 10.20.193.6:0/955522731 conn(0x7efdd40cd600 msgr2=0x7efdd414d030 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 shutdown_connections 2026-02-22T23:28:40.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 -- 10.20.193.6:0/955522731 wait complete. 2026-02-22T23:28:40.438 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.433+0000 7efddb985640 1 librados: shutdown 2026-02-22T23:28:40.442 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2026-02-22T23:28:40.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.508+0000 7f82230b9640 10 monclient: get_monmap_and_config 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.508+0000 7f82230b9640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.508+0000 7f82230b9640 10 monclient: monmap: 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:40.509306+0000 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:40.509306+0000 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:40.510 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:40.511 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:40.511 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:40.511 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:40.511 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:40.511 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 1 Processor -- start 2026-02-22T23:28:40.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 1 -- start start 2026-02-22T23:28:40.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient: init 2026-02-22T23:28:40.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient: _add_conns ranks=[5,4,0,1,3,2] 2026-02-22T23:28:40.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f821c06ff20 0x7f821c070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient(hunting): picked mon.noname-c con 0x7f821c06ff20 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f821c0c99a0 0x7f821c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient(hunting): picked mon.noname-b con 0x7f821c0c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f821c15a410 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f821c15a860 con 0x7f821c15a410 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f821c159a40 con 0x7f821c0c99a0 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f821c0c7e90 con 0x7f821c06ff20 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.509+0000 7f82230b9640 10 monclient(hunting): authenticate will time out at 1395.236673s 2026-02-22T23:28:40.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f822162f640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7f821c15a410 0x7f821c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52246/0 (socket says 10.20.193.6:52246) 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f822162f640 1 -- 10.20.193.6:0/3152228944 learned_addr learned my addr 10.20.193.6:0/3152228944 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 1 --2- 10.20.193.6:0/3152228944 >> v2:10.20.193.40:3300/0 conn(0x7f821c0c99a0 0x7f821c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 1 --2- 10.20.193.6:0/3152228944 >> v2:10.20.193.80:3300/0 conn(0x7f821c06ff20 0x7f821c070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 10 monclient(hunting): get_auth_request con 0x7f821c0c99a0 auth_method 0 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 10 monclient(hunting): get_auth_request con 0x7f821c06ff20 auth_method 0 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8213fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.510+0000 7f8220e2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f82137fe640 1 -- 10.20.193.6:0/3152228944 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3693329734 0 0) 0x7f821c15a860 con 0x7f821c15a410 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f82137fe640 10 monclient(hunting): _negotiate 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f82137fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f82137fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f82137fe640 10 monclient(hunting): my global_id is 5250 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f82137fe640 1 -- 10.20.193.6:0/3152228944 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8208003620 con 0x7f821c15a410 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 10 monclient(hunting): handle_auth_done global_id 5360 payload 274 2026-02-22T23:28:40.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 1 -- 10.20.193.6:0/3152228944 >> v2:10.20.193.40:3300/0 conn(0x7f821c0c99a0 msgr2=0x7f821c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 1 --2- 10.20.193.6:0/3152228944 >> v2:10.20.193.40:3300/0 conn(0x7f821c0c99a0 0x7f821c159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 1 -- 10.20.193.6:0/3152228944 >> v1:10.20.193.6:6789/0 conn(0x7f821c15a410 legacy=0x7f821c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 1 monclient: found mon.noname-c 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.511+0000 7f8220e2e640 1 -- 10.20.193.6:0/3152228944 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c0c8170 con 0x7f821c06ff20 2026-02-22T23:28:40.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.512+0000 7f8220e2e640 1 --2- 10.20.193.6:0/3152228944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c06ff20 0x7f821c070300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f820c0098a0 tx=0x7f820c02ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7816d82c6b6d9 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 1 -- 10.20.193.6:0/3152228944 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c03c070 con 0x7f821c06ff20 2026-02-22T23:28:40.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.525 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: dump: 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.526 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 1 -- 10.20.193.6:0/3152228944 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820c02fae0 con 0x7f821c06ff20 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 5 monclient: authenticate success, global_id 5360 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 1 -- 10.20.193.6:0/3152228944 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c037440 con 0x7f821c06ff20 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 4 set_mon_vals no callback set 2026-02-22T23:28:40.527 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82137fe640 10 monclient: dump: 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.528 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 10 monclient: shutdown 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 -- 10.20.193.6:0/3152228944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c06ff20 msgr2=0x7f821c070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 --2- 10.20.193.6:0/3152228944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c06ff20 0x7f821c070300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f820c0098a0 tx=0x7f820c02ee10 comp rx=0 tx=0).stop 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 -- 10.20.193.6:0/3152228944 shutdown_connections 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 --2- 10.20.193.6:0/3152228944 >> v2:10.20.193.40:3300/0 conn(0x7f821c0c99a0 0x7f821c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 --2- 10.20.193.6:0/3152228944 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c06ff20 0x7f821c070300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 -- 10.20.193.6:0/3152228944 >> 10.20.193.6:0/3152228944 conn(0x7f821c0cd780 msgr2=0x7f821c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 -- 10.20.193.6:0/3152228944 shutdown_connections 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.514+0000 7f82230b9640 1 -- 10.20.193.6:0/3152228944 wait complete. 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 10 monclient: monmap: 2026-02-22T23:28:40.529 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 1 librados: starting msgr at 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 1 librados: starting objecter 2026-02-22T23:28:40.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 1 Processor -- start 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 1 -- start start 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 1 librados: setting wanted keys 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 1 librados: calling monclient init 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.515+0000 7f82230b9640 10 monclient: init 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f821c06ff20 0x7f821c1f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): picked mon.noname-a con 0x7f821c06ff20 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c0c99a0 0x7f821c0c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): picked mon.noname-c con 0x7f821c0c99a0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f821c15a410 0x7f821c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): picked mon.noname-b con 0x7f821c15a410 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:40.531 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f821c0c8a10 con 0x7f821c06ff20 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f821c0c8730 con 0x7f821c15a410 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f821c150520 con 0x7f821c0c99a0 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f82230b9640 10 monclient(hunting): authenticate will time out at 1395.243137s 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f821c15a410 0x7f821c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8213fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c0c99a0 0x7f821c0c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8220e2e640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f821c06ff20 0x7f821c1f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f821c15a410 0x7f821c0c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:43824/0 (socket says 10.20.193.6:43824) 2026-02-22T23:28:40.532 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 1 -- 10.20.193.6:0/2856147715 learned_addr learned my addr 10.20.193.6:0/2856147715 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 10 monclient(hunting): get_auth_request con 0x7f821c15a410 auth_method 0 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8213fff640 10 monclient(hunting): get_auth_request con 0x7f821c0c99a0 auth_method 0 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8213fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8213fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8213fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8220e2e640 10 monclient(hunting): get_auth_request con 0x7f821c06ff20 auth_method 0 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8220e2e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8220e2e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.533 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f8220e2e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.516+0000 7f822162f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.517+0000 7f8213fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.517+0000 7f8213fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.517+0000 7f8213fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.517+0000 7f8220e2e640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.517+0000 7f8220e2e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.517+0000 7f8220e2e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 10 monclient(hunting): handle_auth_done global_id 5212 payload 824 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 1 -- 10.20.193.6:0/2856147715 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c0c99a0 msgr2=0x7f821c0c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c0c99a0 0x7f821c0c10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.534 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 1 -- 10.20.193.6:0/2856147715 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f821c06ff20 msgr2=0x7f821c1f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f821c06ff20 0x7f821c1f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 1 monclient: found mon.noname-b 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.518+0000 7f822162f640 1 -- 10.20.193.6:0/2856147715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c0c1610 con 0x7f821c15a410 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.519+0000 7f822162f640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f821c15a410 0x7f821c0c36e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f821800cb00 tx=0x7f821800a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.519+0000 7f82117fa640 10 client.?.objecter ms_handle_connect 0x7f821c15a410 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.519+0000 7f82117fa640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 1 -- 10.20.193.6:0/2856147715 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821800ac80 con 0x7f821c15a410 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:40.535 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: dump: 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 1 -- 10.20.193.6:0/2856147715 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821800ae20 con 0x7f821c15a410 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 1 -- 10.20.193.6:0/2856147715 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218012600 con 0x7f821c15a410 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f8212ffd640 4 set_mon_vals no callback set 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82117fa640 10 monclient: dump: 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.537 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 5 monclient: authenticate success, global_id 5212 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 10 monclient: _renew_subs 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c0c18e0 con 0x7f821c15a410 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 10 client.5212.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 10 monclient: _renew_subs 2026-02-22T23:28:40.538 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c0c1ea0 con 0x7f821c15a410 2026-02-22T23:28:40.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.521+0000 7f82230b9640 1 librados: init done 2026-02-22T23:28:40.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.522+0000 7f82230b9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.522+0000 7f82230b9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.522+0000 7f82230b9640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.539 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.522+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821c070300 con 0x7f821c15a410 2026-02-22T23:28:40.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.533+0000 7f82117fa640 1 -- 10.20.193.6:0/2856147715 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f82180127a0 con 0x7f821c15a410 2026-02-22T23:28:40.540 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.533+0000 7f82117fa640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81ec078e50 0x7f81ec07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.534+0000 7f8220e2e640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81ec078e50 0x7f81ec07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.534+0000 7f8220e2e640 10 monclient: get_auth_request con 0x7f81ec078e50 auth_method 0 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.534+0000 7f8220e2e640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81ec078e50 0x7f81ec07b310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f820c039000 tx=0x7f820c033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.534+0000 7f82117fa640 10 client.5212.objecter ms_handle_connect 0x7f81ec078e50 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.536+0000 7f82117fa640 1 -- 10.20.193.6:0/2856147715 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f821805e800 con 0x7f821c15a410 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.536+0000 7f82117fa640 10 client.5212.objecter ms_dispatch2 0x7f821c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.536+0000 7f82117fa640 3 client.5212.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.536+0000 7f82117fa640 3 client.5212.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:40.541 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.536+0000 7f82117fa640 20 client.5212.objecter dump_active .. 0 homeless 2026-02-22T23:28:40.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.638+0000 7f82117fa640 1 -- 10.20.193.6:0/2856147715 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f82180627c0 con 0x7f821c15a410 2026-02-22T23:28:40.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.638+0000 7f82117fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.638+0000 7f82117fa640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:40.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.721+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "10", "target": ["mon-mgr", ""]}) -- 0x7f821c0c32b0 con 0x7f81ec078e50 2026-02-22T23:28:40.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.721+0000 7f82117fa640 1 -- 10.20.193.6:0/2856147715 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 10 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7f820c041460 con 0x7f81ec078e50 2026-02-22T23:28:40.722 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 10 to deep-scrub 2026-02-22T23:28:40.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 10 librados: watch_flush enter 2026-02-22T23:28:40.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 10 librados: watch_flush exit 2026-02-22T23:28:40.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 20 client.5212.objecter shutdown clearing up homeless session... 2026-02-22T23:28:40.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 10 client.5212.objecter successfully canceled tick 2026-02-22T23:28:40.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81ec078e50 msgr2=0x7f81ec07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81ec078e50 0x7f81ec07b310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f820c039000 tx=0x7f820c033000 comp rx=0 tx=0).stop 2026-02-22T23:28:40.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 10 monclient: shutdown 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f821c15a410 msgr2=0x7f821c0c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f821c15a410 0x7f821c0c36e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f821800cb00 tx=0x7f821800a060 comp rx=0 tx=0).stop 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 shutdown_connections 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f81ec078e50 0x7f81ec07b310 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f821c15a410 0x7f821c0c36e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f821c0c99a0 0x7f821c0c10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 --2- 10.20.193.6:0/2856147715 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f821c06ff20 0x7f821c1f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 >> 10.20.193.6:0/2856147715 conn(0x7f821c0cd780 msgr2=0x7f821c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 shutdown_connections 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 -- 10.20.193.6:0/2856147715 wait complete. 2026-02-22T23:28:40.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.722+0000 7f82230b9640 1 librados: shutdown 2026-02-22T23:28:40.733 INFO:tasks.ceph:Scrubbing osd.14 2026-02-22T23:28:40.733 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:40.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.802+0000 7fa6ac2bd640 10 monclient: get_monmap_and_config 2026-02-22T23:28:40.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.802+0000 7fa6ac2bd640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.802+0000 7fa6ac2bd640 10 monclient: monmap: 2026-02-22T23:28:40.804 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.804 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:40.803034+0000 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:40.803034+0000 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.802+0000 7fa6ac2bd640 1 Processor -- start 2026-02-22T23:28:40.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.802+0000 7fa6ac2bd640 1 -- start start 2026-02-22T23:28:40.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.802+0000 7fa6ac2bd640 10 monclient: init 2026-02-22T23:28:40.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient: _add_conns ranks=[1,2,4,3,0,5] 2026-02-22T23:28:40.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fa6a40c9130 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:40.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fa6a40c9510 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:40.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa6a40c6340 0x7fa6a415c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient(hunting): picked mon.noname-b con 0x7fa6a40c6340 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a406e3b0 con 0x7fa6a40c9130 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6a406e690 con 0x7fa6a40c9510 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fa6a406e970 con 0x7fa6a40c6340 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6ac2bd640 10 monclient(hunting): authenticate will time out at 1395.530493s 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6aa833640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fa6a40c6340 0x7fa6a415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6a9831640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7fa6a40c9510 0x7fa6a40c5c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:46702/0 (socket says 10.20.193.6:46702) 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.803+0000 7fa6a9831640 1 -- 10.20.193.6:0/3096434137 learned_addr learned my addr 10.20.193.6:0/3096434137 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.804+0000 7fa6aa833640 10 monclient(hunting): get_auth_request con 0x7fa6a40c6340 auth_method 0 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.804+0000 7fa6aa833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.804+0000 7fa6aa833640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.804+0000 7fa6aa833640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.804+0000 7fa6aa833640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.804+0000 7fa6aa833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.804+0000 7fa6aa833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6a9030640 1 -- 10.20.193.6:0/3096434137 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363083245 0 0) 0x7fa6a406e690 con 0x7fa6a40c9510 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6a9030640 10 monclient(hunting): _negotiate 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6a9030640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6a9030640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6a9030640 10 monclient(hunting): my global_id is 5366 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6a9030640 1 -- 10.20.193.6:0/3096434137 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa68c003620 con 0x7fa6a40c9510 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 10 monclient(hunting): handle_auth_done global_id 5215 payload 274 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 1 -- 10.20.193.6:0/3096434137 >> v1:10.20.193.80:6789/0 conn(0x7fa6a40c9510 legacy=0x7fa6a40c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 1 -- 10.20.193.6:0/3096434137 >> v1:10.20.193.40:6789/0 conn(0x7fa6a40c9130 legacy=0x7fa6a406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 1 monclient: found mon.noname-b 2026-02-22T23:28:40.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.805+0000 7fa6aa833640 1 -- 10.20.193.6:0/3096434137 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a406ec50 con 0x7fa6a40c6340 2026-02-22T23:28:40.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.808+0000 7fa6aa833640 1 --2- 10.20.193.6:0/3096434137 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c6340 0x7fa6a415c420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009d90 tx=0x7fa6a002f230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d26e17fd2b7991 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 1 -- 10.20.193.6:0/3096434137 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a003c070 con 0x7fa6a40c6340 2026-02-22T23:28:40.816 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: dump: 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.817 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 5 monclient: authenticate success, global_id 5215 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 1 -- 10.20.193.6:0/3096434137 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a0037440 con 0x7fa6a40c6340 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 1 -- 10.20.193.6:0/3096434137 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0037810 con 0x7fa6a40c6340 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:40.818 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 4 set_mon_vals no callback set 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6a9030640 10 monclient: dump: 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.819 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 10 monclient: shutdown 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3096434137 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c6340 msgr2=0x7fa6a415c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.809+0000 7fa6ac2bd640 1 --2- 10.20.193.6:0/3096434137 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c6340 0x7fa6a415c420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009d90 tx=0x7fa6a002f230 comp rx=0 tx=0).stop 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3096434137 shutdown_connections 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 --2- 10.20.193.6:0/3096434137 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c6340 0x7fa6a415c420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3096434137 >> 10.20.193.6:0/3096434137 conn(0x7fa6a40cd600 msgr2=0x7fa6a40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3096434137 shutdown_connections 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3096434137 wait complete. 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.820 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 10 monclient: monmap: 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 librados: starting msgr at 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 librados: starting objecter 2026-02-22T23:28:40.821 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 Processor -- start 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 -- start start 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 librados: setting wanted keys 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 1 librados: calling monclient init 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.810+0000 7fa6ac2bd640 10 monclient: init 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa6a40c6340 0x7fa6a40bda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): picked mon.noname-c con 0x7fa6a40c6340 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa6a40c9130 0x7fa6a4147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): picked mon.noname-a con 0x7fa6a40c9130 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c9510 0x7fa6a40bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): picked mon.noname-b con 0x7fa6a40c9510 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:40.822 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a415c960 con 0x7fa6a40c9130 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a406f210 con 0x7fa6a40c9510 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a415cae0 con 0x7fa6a40c6340 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6ac2bd640 10 monclient(hunting): authenticate will time out at 1395.538195s 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa6a40c6340 0x7fa6a40bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa833640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c9510 0x7fa6a40bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6a9831640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa6a40c9130 0x7fa6a4147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa6a40c6340 0x7fa6a40bda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43220/0 (socket says 10.20.193.6:43220) 2026-02-22T23:28:40.823 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 1 -- 10.20.193.6:0/3123187265 learned_addr learned my addr 10.20.193.6:0/3123187265 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 10 monclient(hunting): get_auth_request con 0x7fa6a40c6340 auth_method 0 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa833640 10 monclient(hunting): get_auth_request con 0x7fa6a40c9510 auth_method 0 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa833640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa833640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa833640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6a9831640 10 monclient(hunting): get_auth_request con 0x7fa6a40c9130 auth_method 0 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6a9831640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6a9831640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6a9831640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.824 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.811+0000 7fa6aa032640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.812+0000 7fa6aa032640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.812+0000 7fa6aa833640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.812+0000 7fa6aa833640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.812+0000 7fa6aa833640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.812+0000 7fa6a9831640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.812+0000 7fa6a9831640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.812+0000 7fa6a9831640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 10 monclient(hunting): handle_auth_done global_id 5221 payload 824 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 1 -- 10.20.193.6:0/3123187265 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa6a40c6340 msgr2=0x7fa6a40bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fa6a40c6340 0x7fa6a40bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 1 -- 10.20.193.6:0/3123187265 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa6a40c9130 msgr2=0x7fa6a4147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.825 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fa6a40c9130 0x7fa6a4147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 1 monclient: found mon.noname-b 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.813+0000 7fa6aa833640 1 -- 10.20.193.6:0/3123187265 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a40c09a0 con 0x7fa6a40c9510 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.814+0000 7fa6aa833640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c9510 0x7fa6a40bfef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0036000 tx=0x7fa6a0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.814+0000 7fa68affd640 10 client.?.objecter ms_handle_connect 0x7fa6a40c9510 2026-02-22T23:28:40.826 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.814+0000 7fa68affd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:40.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a003c050 con 0x7fa6a40c9510 2026-02-22T23:28:40.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:40.829 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: dump: 2026-02-22T23:28:40.829 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.829 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.830 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a003d040 con 0x7fa6a40c9510 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0041810 con 0x7fa6a40c9510 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 5 monclient: authenticate success, global_id 5221 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 10 monclient: _renew_subs 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a41494f0 con 0x7fa6a40c9510 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6a882f640 4 set_mon_vals no callback set 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa68affd640 10 monclient: dump: 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.831 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 10 client.5221.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 10 monclient: _renew_subs 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.832 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a4149ab0 con 0x7fa6a40c9510 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.815+0000 7fa6ac2bd640 1 librados: init done 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.816+0000 7fa6ac2bd640 10 client.5221.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.816+0000 7fa6ac2bd640 15 client.5221.objecter _session_command_op_assign -1 1 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.816+0000 7fa6ac2bd640 20 client.5221.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.816+0000 7fa6ac2bd640 10 client.5221.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.816+0000 7fa6ac2bd640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.816+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6a40bdfb0 con 0x7fa6a40c9510 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.827+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fa6a00419b0 con 0x7fa6a40c9510 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.827+0000 7fa68affd640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa674078e50 0x7fa67407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.827+0000 7fa6aa032640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa674078e50 0x7fa67407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.833 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.828+0000 7fa6aa032640 10 monclient: get_auth_request con 0x7fa674078e50 auth_method 0 2026-02-22T23:28:40.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.828+0000 7fa6aa032640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa674078e50 0x7fa67407b310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa694002770 tx=0x7fa694002a30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.828+0000 7fa68affd640 10 client.5221.objecter ms_handle_connect 0x7fa674078e50 2026-02-22T23:28:40.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fa6a0083d70 con 0x7fa6a40c9510 2026-02-22T23:28:40.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 10 client.5221.objecter ms_dispatch2 0x7fa6a4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:40.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 3 client.5221.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:40.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 3 client.5221.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 10 client.5221.objecter checking command 1 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7fa674081330 0x7fa674083750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 20 client.5221.objecter _get_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 20 client.5221.objecter put_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 15 client.5221.objecter _session_command_op_remove -1 1 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 20 client.5221.objecter _get_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 20 client.5221.objecter get_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 15 client.5221.objecter _session_command_op_assign 14 1 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 20 client.5221.objecter put_session s=0x7fa674080c60 osd=14 4 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 10 client.5221.objecter _send_command 1 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 --> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa674083e20 con 0x7fa674081330 2026-02-22T23:28:40.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.829+0000 7fa68affd640 20 client.5221.objecter dump_active .. 0 homeless 2026-02-22T23:28:40.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.830+0000 7fa6a9831640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7fa674081330 0x7fa674083750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.830+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6a0084130 con 0x7fa6a40c9510 2026-02-22T23:28:40.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.830+0000 7fa68affd640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:40.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.830+0000 7fa6a9831640 10 monclient: get_auth_request con 0x7fa674081330 auth_method 0 2026-02-22T23:28:40.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.832+0000 7fa68affd640 10 client.5221.objecter ms_handle_connect 0x7fa674081330 2026-02-22T23:28:40.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.832+0000 7fa6a9831640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7fa674081330 0x7fa674083750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.14 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.841+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== osd.14 v2:10.20.193.80:6824/3509265050 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fa674083e20 con 0x7fa674081330 2026-02-22T23:28:40.842 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.841+0000 7fa68affd640 10 client.5221.objecter ms_dispatch2 0x7fa6a4000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:40.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.841+0000 7fa68affd640 10 client.5221.objecter _finish_command 1 = system:0 2026-02-22T23:28:40.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.841+0000 7fa68affd640 20 client.5221.objecter put_session s=0x7fa674080c60 osd=14 4 2026-02-22T23:28:40.843 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.841+0000 7fa68affd640 15 client.5221.objecter _session_command_op_remove 14 1 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 10 client.5221.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 15 client.5221.objecter _session_command_op_assign -1 2 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 20 client.5221.objecter _get_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 20 client.5221.objecter put_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 20 client.5221.objecter _get_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 15 client.5221.objecter _session_command_op_remove -1 2 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 20 client.5221.objecter get_session s=0x7fa674080c60 osd=14 3 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 15 client.5221.objecter _session_command_op_assign 14 2 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 20 client.5221.objecter put_session s=0x7fa674080c60 osd=14 4 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 10 client.5221.objecter _send_command 2 2026-02-22T23:28:40.849 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.847+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 --> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa6a415c420 con 0x7fa674081330 2026-02-22T23:28:40.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa68affd640 1 -- 10.20.193.6:0/3123187265 <== osd.14 v2:10.20.193.80:6824/3509265050 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa6a415c420 con 0x7fa674081330 2026-02-22T23:28:40.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa68affd640 10 client.5221.objecter ms_dispatch2 0x7fa6a4000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:40.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa68affd640 10 client.5221.objecter _finish_command 2 = system:0 2026-02-22T23:28:40.855 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa68affd640 20 client.5221.objecter put_session s=0x7fa674080c60 osd=14 4 2026-02-22T23:28:40.856 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa68affd640 15 client.5221.objecter _session_command_op_remove 14 2 2026-02-22T23:28:40.856 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:40.856 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:40.856 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 10 librados: watch_flush enter 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 10 librados: watch_flush exit 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 10 client.5221.objecter close_session for osd.14 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7fa674081330 msgr2=0x7fa674083750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.80:6824/3509265050,v1:10.20.193.80:6825/3509265050] conn(0x7fa674081330 0x7fa674083750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 20 client.5221.objecter put_session s=0x7fa674080c60 osd=14 1 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 20 client.5221.objecter shutdown clearing up homeless session... 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 10 client.5221.objecter successfully canceled tick 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa674078e50 msgr2=0x7fa67407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fa674078e50 0x7fa67407b310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa694002770 tx=0x7fa694002a30 comp rx=0 tx=0).stop 2026-02-22T23:28:40.858 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 10 monclient: shutdown 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c9510 msgr2=0x7fa6a40bfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 --2- 10.20.193.6:0/3123187265 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fa6a40c9510 0x7fa6a40bfef0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0036000 tx=0x7fa6a0004b40 comp rx=0 tx=0).stop 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6aa833640 1 -- 10.20.193.6:0/3123187265 reap_dead start 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 shutdown_connections 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 >> 10.20.193.6:0/3123187265 conn(0x7fa6a40cd600 msgr2=0x7fa6a406fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 shutdown_connections 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 -- 10.20.193.6:0/3123187265 wait complete. 2026-02-22T23:28:40.859 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.854+0000 7fa6ac2bd640 1 librados: shutdown 2026-02-22T23:28:40.863 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2026-02-22T23:28:40.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.931+0000 7febdd549640 10 monclient: get_monmap_and_config 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.931+0000 7febdd549640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.931+0000 7febdd549640 10 monclient: monmap: 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:40.931824+0000 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:40.931824+0000 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.933 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.931+0000 7febdd549640 1 Processor -- start 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.931+0000 7febdd549640 1 -- start start 2026-02-22T23:28:40.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.931+0000 7febdd549640 10 monclient: init 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 10 monclient: _add_conns ranks=[1,2,0,5,3,4] 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7febd806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7febd80c99a0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7febd815a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd80c7e90 con 0x7febd815a5e0 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd80c8170 con 0x7febd806ff20 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd80c8450 con 0x7febd80c99a0 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.941 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febdd549640 10 monclient(hunting): authenticate will time out at 1395.659299s 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febd6ffd640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7febd806ff20 0x7febd8159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38360/0 (socket says 10.20.193.6:38360) 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febd77fe640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7febd815a5e0 0x7febd80c6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52258/0 (socket says 10.20.193.6:52258) 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.932+0000 7febd6ffd640 1 -- 10.20.193.6:0/3139174624 learned_addr learned my addr 10.20.193.6:0/3139174624 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2988004424 0 0) 0x7febd80c7e90 con 0x7febd815a5e0 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): my global_id is 5259 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febc4003620 con 0x7febd815a5e0 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1167080928 0 0) 0x7febd80c8450 con 0x7febd80c99a0 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:40.942 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): my global_id is 5372 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febd80c7e90 con 0x7febd80c99a0 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1568060259 0 0) 0x7febd80c8170 con 0x7febd806ff20 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 10 monclient(hunting): my global_id is 5224 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.934+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febd80c8450 con 0x7febd806ff20 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3785656083 0 0) 0x7febd80c7e90 con 0x7febd80c99a0 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 >> v1:10.20.193.40:6789/0 conn(0x7febd806ff20 legacy=0x7febd8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.943 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 >> v1:10.20.193.6:6789/0 conn(0x7febd815a5e0 legacy=0x7febd80c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd80c8730 con 0x7febd80c99a0 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7febc0002ee0 con 0x7febd80c99a0 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febdd549640 5 monclient: authenticate success, global_id 5372 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febdd549640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.944 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: dump: 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.945 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 >> v1:10.20.193.80:6789/0 conn(0x7febd80c99a0 legacy=0x7febd814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febc4005040 0x7febc40038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): picked mon.b con 0x7febc4005040 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febc4003de0 0x7febc4004260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): picked mon.a con 0x7febc4003de0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4009ab0 0x7febc400bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): picked mon.c con 0x7febc4009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7febc400c570 con 0x7febc4003de0 2026-02-22T23:28:40.946 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7febc400c6f0 con 0x7febc4005040 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7febc400c8f0 con 0x7febc4009ab0 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febd5ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.935+0000 7febdd549640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febc4005040 0x7febc40038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febc4003de0 0x7febc4004260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 10 monclient(hunting): get_auth_request con 0x7febc4005040 auth_method 0 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.947 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 10 monclient(hunting): get_auth_request con 0x7febc4003de0 auth_method 0 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd77fe640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4009ab0 0x7febc400bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.936+0000 7febd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 10 monclient(hunting): get_auth_request con 0x7febc4009ab0 auth_method 0 2026-02-22T23:28:40.948 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 10 monclient(hunting): handle_auth_done global_id 5372 payload 293 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 1 -- 10.20.193.6:0/3139174624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4009ab0 msgr2=0x7febc400bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4009ab0 0x7febc400bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 1 -- 10.20.193.6:0/3139174624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febc4005040 msgr2=0x7febc40038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febc4005040 0x7febc40038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 1 monclient: found mon.a 2026-02-22T23:28:40.949 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 1 -- 10.20.193.6:0/3139174624 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd80c7e90 con 0x7febc4003de0 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd77fe640 1 -- 10.20.193.6:0/3139174624 reap_dead start 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.937+0000 7febd6ffd640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febc4005040 0x7febc40038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.938+0000 7febd67fc640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febc4003de0 0x7febc4004260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7febc0007f90 tx=0x7febc002f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c5ca976fdb2adf server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.939+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc003c070 con 0x7febc4003de0 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.939+0000 7febd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.939+0000 7febd5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.939+0000 7febd5ffb640 10 monclient: dump: 2026-02-22T23:28:40.950 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.951 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.939+0000 7febdd549640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:40.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc002fe20 con 0x7febc4003de0 2026-02-22T23:28:40.957 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febd5ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febd5ffb640 1 -- 10.20.193.6:0/3139174624 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0007bb0 con 0x7febc4003de0 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febdd549640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febdd549640 4 set_mon_vals no callback set 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febd5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.940+0000 7febd5ffb640 10 monclient: dump: 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.958 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 10 monclient: shutdown 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 -- 10.20.193.6:0/3139174624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febc4003de0 msgr2=0x7febc4004260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febc4003de0 0x7febc4004260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7febc0007f90 tx=0x7febc002f3e0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 -- 10.20.193.6:0/3139174624 shutdown_connections 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 --2- 10.20.193.6:0/3139174624 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febc4003de0 0x7febc4004260 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 -- 10.20.193.6:0/3139174624 >> 10.20.193.6:0/3139174624 conn(0x7febd80cd780 msgr2=0x7febd80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:40.959 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 -- 10.20.193.6:0/3139174624 shutdown_connections 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 -- 10.20.193.6:0/3139174624 wait complete. 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 10 monclient: build_initial_monmap 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 10 monclient: monmap: 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:40.960 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 librados: starting msgr at 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 librados: starting objecter 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 Processor -- start 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 -- start start 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 librados: setting wanted keys 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 1 librados: calling monclient init 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.941+0000 7febdd549640 10 monclient: init 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:28:40.961 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4003de0 0x7febd81f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): picked mon.noname-c con 0x7febc4003de0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febd81f2240 0x7febd80cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): picked mon.noname-b con 0x7febd81f2240 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febd80cc7f0 0x7febd80c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): picked mon.noname-a con 0x7febd80cc7f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7febd8070300 con 0x7febd80cc7f0 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7febd8159a40 con 0x7febd81f2240 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7febd8070480 con 0x7febc4003de0 2026-02-22T23:28:40.962 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febdd549640 10 monclient(hunting): authenticate will time out at 1395.669304s 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd77fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febd80cc7f0 0x7febd80c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd6ffd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4003de0 0x7febd81f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd67fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febd81f2240 0x7febd80cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd77fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febd80cc7f0 0x7febd80c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37094/0 (socket says 10.20.193.6:37094) 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd77fe640 1 -- 10.20.193.6:0/3994131592 learned_addr learned my addr 10.20.193.6:0/3994131592 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd77fe640 10 monclient(hunting): get_auth_request con 0x7febd80cc7f0 auth_method 0 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd6ffd640 10 monclient(hunting): get_auth_request con 0x7febc4003de0 auth_method 0 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.963 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd67fc640 10 monclient(hunting): get_auth_request con 0x7febd81f2240 auth_method 0 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.942+0000 7febd67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:40.964 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.943+0000 7febd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 10 monclient(hunting): handle_auth_done global_id 5227 payload 824 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 1 -- 10.20.193.6:0/3994131592 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4003de0 msgr2=0x7febd81f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4003de0 0x7febd81f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 1 -- 10.20.193.6:0/3994131592 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febd80cc7f0 msgr2=0x7febd80c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febd80cc7f0 0x7febd80c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 1 monclient: found mon.noname-b 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.944+0000 7febd67fc640 1 -- 10.20.193.6:0/3994131592 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd80cb210 con 0x7febd81f2240 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.945+0000 7febd67fc640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febd81f2240 0x7febd80cc2b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febc0002970 tx=0x7febd80c8900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.965 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.945+0000 7febb77fe640 10 client.?.objecter ms_handle_connect 0x7febd81f2240 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.945+0000 7febb77fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 1 -- 10.20.193.6:0/3994131592 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0045020 con 0x7febd81f2240 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: dump: 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.966 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 1 -- 10.20.193.6:0/3994131592 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc0002ce0 con 0x7febd81f2240 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 5 monclient: authenticate success, global_id 5227 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 monclient: _renew_subs 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd80cb4f0 con 0x7febd81f2240 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 1 -- 10.20.193.6:0/3994131592 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc003c040 con 0x7febd81f2240 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febb77fe640 10 monclient: dump: 2026-02-22T23:28:40.967 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febd57fa640 4 set_mon_vals no callback set 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 client.5227.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 monclient: _renew_subs 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.968 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd80cbab0 con 0x7febd81f2240 2026-02-22T23:28:40.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 1 librados: init done 2026-02-22T23:28:40.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:40.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:40.969 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.947+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8159ed0 con 0x7febd81f2240 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.958+0000 7febb77fe640 1 -- 10.20.193.6:0/3994131592 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7febc00036f0 con 0x7febd81f2240 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.958+0000 7febb77fe640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feba4078e50 0x7feba407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.958+0000 7febd6ffd640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feba4078e50 0x7feba407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.958+0000 7febd6ffd640 10 monclient: get_auth_request con 0x7feba4078e50 auth_method 0 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.959+0000 7febb77fe640 10 client.5227.objecter ms_handle_connect 0x7feba4078e50 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.959+0000 7febd6ffd640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feba4078e50 0x7feba407b310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7febcc002410 tx=0x7febcc002e10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.959+0000 7febb77fe640 1 -- 10.20.193.6:0/3994131592 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7febc0007bb0 con 0x7febd81f2240 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.959+0000 7febb77fe640 10 client.5227.objecter ms_dispatch2 0x7febd8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.959+0000 7febb77fe640 3 client.5227.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.959+0000 7febb77fe640 3 client.5227.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:40.970 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:40.959+0000 7febb77fe640 20 client.5227.objecter dump_active .. 0 homeless 2026-02-22T23:28:41.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.060+0000 7febb77fe640 1 -- 10.20.193.6:0/3994131592 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7febc002fe20 con 0x7febd81f2240 2026-02-22T23:28:41.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.060+0000 7febb77fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.061 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.060+0000 7febb77fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.143+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "14", "target": ["mon-mgr", ""]}) -- 0x7febd81585a0 con 0x7feba4078e50 2026-02-22T23:28:41.145 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febb77fe640 1 -- 10.20.193.6:0/3994131592 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 14 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7febd81585a0 con 0x7feba4078e50 2026-02-22T23:28:41.145 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 14 to deep-scrub 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 10 librados: watch_flush enter 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 10 librados: watch_flush exit 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 20 client.5227.objecter shutdown clearing up homeless session... 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 10 client.5227.objecter successfully canceled tick 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feba4078e50 msgr2=0x7feba407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feba4078e50 0x7feba407b310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7febcc002410 tx=0x7febcc002e10 comp rx=0 tx=0).stop 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 10 monclient: shutdown 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febd81f2240 msgr2=0x7febd80cc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.147 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.144+0000 7febdd549640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febd81f2240 0x7febd80cc2b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febc0002970 tx=0x7febd80c8900 comp rx=0 tx=0).stop 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 shutdown_connections 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7feba4078e50 0x7feba407b310 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7febd80cc7f0 0x7febd80c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7febd81f2240 0x7febd80cc2b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 --2- 10.20.193.6:0/3994131592 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7febc4003de0 0x7febd81f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 >> 10.20.193.6:0/3994131592 conn(0x7febd80cd780 msgr2=0x7febd8145b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 shutdown_connections 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 -- 10.20.193.6:0/3994131592 wait complete. 2026-02-22T23:28:41.148 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.145+0000 7febdd549640 1 librados: shutdown 2026-02-22T23:28:41.155 INFO:tasks.ceph:Scrubbing osd.3 2026-02-22T23:28:41.155 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:41.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.222+0000 7f0bdd2fc640 10 monclient: get_monmap_and_config 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.222+0000 7f0bdd2fc640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.222+0000 7f0bdd2fc640 10 monclient: monmap: 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:41.222849+0000 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:41.222849+0000 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.224 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.222+0000 7f0bdd2fc640 1 Processor -- start 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.222+0000 7f0bdd2fc640 1 -- start start 2026-02-22T23:28:41.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.222+0000 7f0bdd2fc640 10 monclient: init 2026-02-22T23:28:41.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 10 monclient: _add_conns ranks=[1,2,0,5,3,4] 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f0bd81503e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f0bd806fdb0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f0bd80c8aa0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd815a570 con 0x7f0bd80c8aa0 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd815a850 con 0x7f0bd81503e0 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd80c7fc0 con 0x7f0bd806fdb0 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bdd2fc640 10 monclient(hunting): authenticate will time out at 1395.950220s 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bd6ffd640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f0bd81503e0 0x7f0bd8159b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38362/0 (socket says 10.20.193.6:38362) 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bd6ffd640 1 -- 10.20.193.6:0/1401533125 learned_addr learned my addr 10.20.193.6:0/1401533125 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:41.232 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.223+0000 7f0bd77fe640 1 --1- 10.20.193.6:0/1401533125 >> v1:10.20.193.6:6789/0 conn(0x7f0bd80c8aa0 0x7f0bd80c8f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52270/0 (socket says 10.20.193.6:52270) 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.224+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291349852 0 0) 0x7f0bd815a850 con 0x7f0bd81503e0 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.224+0000 7f0bd5ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.224+0000 7f0bd5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.224+0000 7f0bd5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.224+0000 7f0bd5ffb640 10 monclient(hunting): my global_id is 5230 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bc4003620 con 0x7f0bd81503e0 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3436118379 0 0) 0x7f0bd80c7fc0 con 0x7f0bd806fdb0 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): my global_id is 5378 2026-02-22T23:28:41.233 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bd815a850 con 0x7f0bd806fdb0 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795645322 0 0) 0x7f0bd815a570 con 0x7f0bd80c8aa0 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 10 monclient(hunting): my global_id is 5265 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.225+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bd80c7fc0 con 0x7f0bd80c8aa0 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.1 v1:10.20.193.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1172679166 0 0) 0x7f0bc4003620 con 0x7f0bd81503e0 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 >> v1:10.20.193.80:6789/0 conn(0x7f0bd806fdb0 legacy=0x7f0bd80701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 >> v1:10.20.193.6:6789/0 conn(0x7f0bd80c8aa0 legacy=0x7f0bd80c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 monclient: found mon.noname-b-legacy 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.234 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: _send_mon_message to mon.noname-b-legacy at v1:10.20.193.40:6789/0 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 --> v1:10.20.193.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd80c82a0 con 0x7f0bd81503e0 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.1 v1:10.20.193.40:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7f0bcc002f00 con 0x7f0bd81503e0 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bdd2fc640 5 monclient: authenticate success, global_id 5230 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bdd2fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: got monmap 1 from mon.noname-b-legacy (according to old e1) 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: dump: 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.235 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 monclient: mon.b has (v2) addrs [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] but i'm connected to v1:10.20.193.40:6789/0, reconnecting 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 >> v1:10.20.193.40:6789/0 conn(0x7f0bd81503e0 legacy=0x7f0bd8159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.236 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient: _add_conns ranks=[2,0,1] 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bc4003fe0 0x7f0bc4004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): picked mon.c con 0x7f0bc4003fe0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4004e40 0x7f0bc40052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): picked mon.a con 0x7f0bc4004e40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bc4009ab0 0x7f0bc400bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): picked mon.b con 0x7f0bc4009ab0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc400c490 con 0x7f0bc4004e40 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc400c640 con 0x7f0bc4009ab0 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc400c840 con 0x7f0bc4003fe0 2026-02-22T23:28:41.237 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bd5ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.226+0000 7f0bdd2fc640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.227+0000 7f0bd6ffd640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bc4003fe0 0x7f0bc4004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.227+0000 7f0bd67fc640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4004e40 0x7f0bc40052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.227+0000 7f0bd77fe640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bc4009ab0 0x7f0bc400bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.227+0000 7f0bd6ffd640 10 monclient(hunting): get_auth_request con 0x7f0bc4003fe0 auth_method 0 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.227+0000 7f0bd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.227+0000 7f0bd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.227+0000 7f0bd6ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd67fc640 10 monclient(hunting): get_auth_request con 0x7f0bc4004e40 auth_method 0 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.238 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd67fc640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd77fe640 10 monclient(hunting): get_auth_request con 0x7f0bc4009ab0 auth_method 0 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd77fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.239 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.228+0000 7f0bd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 10 monclient(hunting): handle_auth_done global_id 5230 payload 293 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 1 -- 10.20.193.6:0/1401533125 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bc4009ab0 msgr2=0x7f0bc400bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bc4009ab0 0x7f0bc400bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 1 -- 10.20.193.6:0/1401533125 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4004e40 msgr2=0x7f0bc40052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4004e40 0x7f0bc40052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 1 monclient: found mon.c 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 1 -- 10.20.193.6:0/1401533125 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc4003620 con 0x7f0bc4003fe0 2026-02-22T23:28:41.240 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd77fe640 1 -- 10.20.193.6:0/1401533125 reap_dead start 2026-02-22T23:28:41.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.229+0000 7f0bd6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.241 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.230+0000 7f0bd6ffd640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bc4003fe0 0x7f0bc4004900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c1c0 tx=0x7f0bcc02f540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9a5fd279abdc18 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.246 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007d00 con 0x7f0bc4003fe0 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 10 monclient: dump: 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.247 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bcc005bf0 con 0x7f0bc4003fe0 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bdd2fc640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 1 -- 10.20.193.6:0/1401533125 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007070 con 0x7f0bc4003fe0 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bd5ffb640 10 monclient: dump: 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.248 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bdd2fc640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bdd2fc640 4 set_mon_vals no callback set 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bdd2fc640 10 monclient: shutdown 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bdd2fc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.249 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/1401533125 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bc4003fe0 msgr2=0x7f0bc4004900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.231+0000 7f0bdd2fc640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bc4003fe0 0x7f0bc4004900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c1c0 tx=0x7f0bcc02f540 comp rx=0 tx=0).stop 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/1401533125 shutdown_connections 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 --2- 10.20.193.6:0/1401533125 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bc4003fe0 0x7f0bc4004900 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/1401533125 >> 10.20.193.6:0/1401533125 conn(0x7f0bd80cd600 msgr2=0x7f0bd80cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/1401533125 shutdown_connections 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/1401533125 wait complete. 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 10 monclient: monmap: 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:41.250 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 librados: starting msgr at 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 librados: starting objecter 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 Processor -- start 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 -- start start 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 librados: setting wanted keys 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 1 librados: calling monclient init 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.232+0000 7f0bdd2fc640 10 monclient: init 2026-02-22T23:28:41.251 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4003fe0 0x7f0bd81f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): picked mon.noname-a con 0x7f0bc4003fe0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bd81f3f30 0x7f0bd806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): picked mon.noname-c con 0x7f0bd81f3f30 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bd806f3c0 0x7f0bd814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): picked mon.noname-b con 0x7f0bd806f3c0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd80c7d70 con 0x7f0bc4003fe0 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81509d0 con 0x7f0bd806f3c0 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.252 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd80c8610 con 0x7f0bd81f3f30 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bdd2fc640 10 monclient(hunting): authenticate will time out at 1395.960420s 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd6ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4003fe0 0x7f0bd81f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd67fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bd81f3f30 0x7f0bd806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd6ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4003fe0 0x7f0bd81f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37104/0 (socket says 10.20.193.6:37104) 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd67fc640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bd81f3f30 0x7f0bd806ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43258/0 (socket says 10.20.193.6:43258) 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd6ffd640 1 -- 10.20.193.6:0/319023625 learned_addr learned my addr 10.20.193.6:0/319023625 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd6ffd640 10 monclient(hunting): get_auth_request con 0x7f0bc4003fe0 auth_method 0 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd6ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd6ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd6ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.253 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd67fc640 10 monclient(hunting): get_auth_request con 0x7f0bd81f3f30 auth_method 0 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd67fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd67fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd67fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.233+0000 7f0bd77fe640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bd806f3c0 0x7f0bd814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd67fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd67fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd67fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd6ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd6ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd77fe640 10 monclient(hunting): get_auth_request con 0x7f0bd806f3c0 auth_method 0 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd77fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.254 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd77fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd77fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd77fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd77fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.234+0000 7f0bd77fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 10 monclient(hunting): handle_auth_done global_id 5381 payload 824 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 1 -- 10.20.193.6:0/319023625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bd806f3c0 msgr2=0x7f0bd814afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f0bd806f3c0 0x7f0bd814afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 1 -- 10.20.193.6:0/319023625 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4003fe0 msgr2=0x7f0bd81f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f0bc4003fe0 0x7f0bd81f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 1 monclient: found mon.noname-c 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.255 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.235+0000 7f0bd67fc640 1 -- 10.20.193.6:0/319023625 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd81f7a40 con 0x7f0bd81f3f30 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.236+0000 7f0bd67fc640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bd81f3f30 0x7f0bd806ee80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000ead0 tx=0x7f0bc00056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.236+0000 7f0bb77fe640 10 client.?.objecter ms_handle_connect 0x7f0bd81f3f30 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.236+0000 7f0bb77fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0019070 con 0x7f0bd81f3f30 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: dump: 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.256 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0006ba0 con 0x7f0bd81f3f30 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000c820 con 0x7f0bd81f3f30 2026-02-22T23:28:41.257 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bd57fa640 4 set_mon_vals no callback set 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bb77fe640 10 monclient: dump: 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.258 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bdd2fc640 5 monclient: authenticate success, global_id 5381 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bdd2fc640 10 monclient: _renew_subs 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bdd2fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.237+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd81f3770 con 0x7f0bd81f3f30 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 10 client.5381.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 10 monclient: _renew_subs 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd81f7480 con 0x7f0bd81f3f30 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 1 librados: init done 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 10 client.5381.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 15 client.5381.objecter _session_command_op_assign -1 1 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 20 client.5381.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 10 client.5381.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.259 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.238+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0bd80701d0 con 0x7f0bd81f3f30 2026-02-22T23:28:41.261 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.249+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f0bc0003ce0 con 0x7f0bd81f3f30 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.250+0000 7f0bb77fe640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0ba4078e00 0x7f0ba407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.250+0000 7f0bd6ffd640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0ba4078e00 0x7f0ba407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.250+0000 7f0bd6ffd640 10 monclient: get_auth_request con 0x7f0ba4078e00 auth_method 0 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.250+0000 7f0bd6ffd640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0ba4078e00 0x7f0ba407b2c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0027c0 tx=0x7f0bcc009070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.250+0000 7f0bb77fe640 10 client.5381.objecter ms_handle_connect 0x7f0ba4078e00 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f0bc00972a0 con 0x7f0bd81f3f30 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 10 client.5381.objecter ms_dispatch2 0x7f0bd8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 3 client.5381.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 3 client.5381.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 10 client.5381.objecter checking command 1 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f0ba40812e0 0x7f0ba4083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 20 client.5381.objecter _get_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 20 client.5381.objecter put_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.262 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 15 client.5381.objecter _session_command_op_remove -1 1 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 20 client.5381.objecter _get_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 20 client.5381.objecter get_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 15 client.5381.objecter _session_command_op_assign 3 1 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 20 client.5381.objecter put_session s=0x7f0ba4080c10 osd=3 4 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 10 client.5381.objecter _send_command 1 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ba4083dd0 con 0x7f0ba40812e0 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.251+0000 7f0bb77fe640 20 client.5381.objecter dump_active .. 0 homeless 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.252+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0bc005de70 con 0x7f0bd81f3f30 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.252+0000 7f0bb77fe640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.252+0000 7f0bd77fe640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f0ba40812e0 0x7f0ba4083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.252+0000 7f0bd77fe640 10 monclient: get_auth_request con 0x7f0ba40812e0 auth_method 0 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.254+0000 7f0bd77fe640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f0ba40812e0 0x7f0ba4083700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.263 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.254+0000 7f0bb77fe640 10 client.5381.objecter ms_handle_connect 0x7f0ba40812e0 2026-02-22T23:28:41.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.264+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== osd.3 v2:10.20.193.189:6818/2996878330 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f0ba4083dd0 con 0x7f0ba40812e0 2026-02-22T23:28:41.265 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.264+0000 7f0bb77fe640 10 client.5381.objecter ms_dispatch2 0x7f0bd8000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:41.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.264+0000 7f0bb77fe640 10 client.5381.objecter _finish_command 1 = system:0 2026-02-22T23:28:41.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.264+0000 7f0bb77fe640 20 client.5381.objecter put_session s=0x7f0ba4080c10 osd=3 4 2026-02-22T23:28:41.266 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.264+0000 7f0bb77fe640 15 client.5381.objecter _session_command_op_remove 3 1 2026-02-22T23:28:41.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 10 client.5381.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:41.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 15 client.5381.objecter _session_command_op_assign -1 2 2026-02-22T23:28:41.271 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 20 client.5381.objecter _get_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 20 client.5381.objecter put_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 20 client.5381.objecter _get_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 15 client.5381.objecter _session_command_op_remove -1 2 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 20 client.5381.objecter get_session s=0x7f0ba4080c10 osd=3 3 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 15 client.5381.objecter _session_command_op_assign 3 2 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 20 client.5381.objecter put_session s=0x7f0ba4080c10 osd=3 4 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 10 client.5381.objecter _send_command 2 2026-02-22T23:28:41.272 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.270+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 --> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0bd814f950 con 0x7f0ba40812e0 2026-02-22T23:28:41.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bb77fe640 1 -- 10.20.193.6:0/319023625 <== osd.3 v2:10.20.193.189:6818/2996878330 2 ==== command_reply(tid 2: 0 ) ==== 8+0+712 (crc 0 0 0) 0x7f0bd814f950 con 0x7f0ba40812e0 2026-02-22T23:28:41.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bb77fe640 10 client.5381.objecter ms_dispatch2 0x7f0bd8000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:41.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bb77fe640 10 client.5381.objecter _finish_command 2 = system:0 2026-02-22T23:28:41.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bb77fe640 20 client.5381.objecter put_session s=0x7f0ba4080c10 osd=3 4 2026-02-22T23:28:41.285 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bb77fe640 15 client.5381.objecter _session_command_op_remove 3 2 2026-02-22T23:28:41.286 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:41.286 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T23:28:41.286 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:41.287 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 10 librados: watch_flush enter 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 10 librados: watch_flush exit 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 10 client.5381.objecter close_session for osd.3 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f0ba40812e0 msgr2=0x7f0ba4083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6818/2996878330,v1:10.20.193.189:6819/2996878330] conn(0x7f0ba40812e0 0x7f0ba4083700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 20 client.5381.objecter put_session s=0x7f0ba4080c10 osd=3 1 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 20 client.5381.objecter shutdown clearing up homeless session... 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 10 client.5381.objecter successfully canceled tick 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0ba4078e00 msgr2=0x7f0ba407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f0ba4078e00 0x7f0ba407b2c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0027c0 tx=0x7f0bcc009070 comp rx=0 tx=0).stop 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 10 monclient: shutdown 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.288 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bd81f3f30 msgr2=0x7f0bd806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 --2- 10.20.193.6:0/319023625 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f0bd81f3f30 0x7f0bd806ee80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000ead0 tx=0x7f0bc00056b0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bd77fe640 1 -- 10.20.193.6:0/319023625 reap_dead start 2026-02-22T23:28:41.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 shutdown_connections 2026-02-22T23:28:41.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 >> 10.20.193.6:0/319023625 conn(0x7f0bd80cd600 msgr2=0x7f0bd80cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 shutdown_connections 2026-02-22T23:28:41.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 -- 10.20.193.6:0/319023625 wait complete. 2026-02-22T23:28:41.289 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.284+0000 7f0bdd2fc640 1 librados: shutdown 2026-02-22T23:28:41.294 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-22T23:28:41.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.362+0000 7f8ff9fe3640 10 monclient: get_monmap_and_config 2026-02-22T23:28:41.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.362+0000 7f8ff9fe3640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.364 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.363+0000 7f8ff9fe3640 10 monclient: monmap: 2026-02-22T23:28:41.364 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.364 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:41.363610+0000 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:41.363610+0000 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:41.365 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.363+0000 7f8ff9fe3640 1 Processor -- start 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.363+0000 7f8ff9fe3640 1 -- start start 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.363+0000 7f8ff9fe3640 10 monclient: init 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient: _add_conns ranks=[2,5,1,3,4,0] 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f8ff406ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8ff40c99a0 0x7f8ff4070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.368 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient(hunting): picked mon.noname-c con 0x7f8ff40c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f8ff415a5e0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff40c7e90 con 0x7f8ff415a5e0 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff40c8170 con 0x7f8ff406ff20 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff4159a40 con 0x7f8ff40c99a0 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff9fe3640 10 monclient(hunting): authenticate will time out at 1396.091087s 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff2ffd640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f8ff40c99a0 0x7f8ff4070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff3fff640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7f8ff415a5e0 0x7f8ff414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38376/0 (socket says 10.20.193.6:38376) 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff3fff640 1 -- 10.20.193.6:0/2193313261 learned_addr learned my addr 10.20.193.6:0/2193313261 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff2ffd640 10 monclient(hunting): get_auth_request con 0x7f8ff40c99a0 auth_method 0 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.369 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.364+0000 7f8ff2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.365+0000 7f8ff2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.365+0000 7f8ff2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.365+0000 7f8ff2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff27fc640 1 -- 10.20.193.6:0/2193313261 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865224736 0 0) 0x7f8ff40c7e90 con 0x7f8ff415a5e0 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff27fc640 10 monclient(hunting): _negotiate 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff27fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff27fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff27fc640 10 monclient(hunting): my global_id is 5236 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff27fc640 1 -- 10.20.193.6:0/2193313261 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fd8003620 con 0x7f8ff415a5e0 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 10 monclient(hunting): handle_auth_done global_id 5384 payload 274 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 1 -- 10.20.193.6:0/2193313261 >> v1:10.20.193.80:6789/0 conn(0x7f8ff406ff20 legacy=0x7f8ff4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.370 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 1 -- 10.20.193.6:0/2193313261 >> v1:10.20.193.40:6789/0 conn(0x7f8ff415a5e0 legacy=0x7f8ff414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 1 monclient: found mon.noname-c 2026-02-22T23:28:41.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.366+0000 7f8ff2ffd640 1 -- 10.20.193.6:0/2193313261 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff40c8450 con 0x7f8ff40c99a0 2026-02-22T23:28:41.371 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.368+0000 7f8ff2ffd640 1 --2- 10.20.193.6:0/2193313261 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff40c99a0 0x7f8ff4070300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098d0 tx=0x7f8fe402eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=311477c82936ad74 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.376 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 1 -- 10.20.193.6:0/2193313261 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe403c070 con 0x7f8ff40c99a0 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: dump: 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.377 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 1 -- 10.20.193.6:0/2193313261 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe402fba0 con 0x7f8ff40c99a0 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 1 -- 10.20.193.6:0/2193313261 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4037440 con 0x7f8ff40c99a0 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff9fe3640 5 monclient: authenticate success, global_id 5384 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff9fe3640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff9fe3640 4 set_mon_vals no callback set 2026-02-22T23:28:41.378 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.371+0000 7f8ff27fc640 10 monclient: dump: 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.379 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 10 monclient: shutdown 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/2193313261 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff40c99a0 msgr2=0x7f8ff4070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/2193313261 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff40c99a0 0x7f8ff4070300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40098d0 tx=0x7f8fe402eed0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/2193313261 shutdown_connections 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/2193313261 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff40c99a0 0x7f8ff4070300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/2193313261 >> 10.20.193.6:0/2193313261 conn(0x7f8ff40cd780 msgr2=0x7f8ff40cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/2193313261 shutdown_connections 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/2193313261 wait complete. 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 10 monclient: monmap: 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.380 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 librados: starting msgr at 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 librados: starting objecter 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 Processor -- start 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 -- start start 2026-02-22T23:28:41.381 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 librados: setting wanted keys 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 1 librados: calling monclient init 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.372+0000 7f8ff9fe3640 10 monclient: init 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff406ff20 0x7f8ff40bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): picked mon.noname-b con 0x7f8ff406ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff40c99a0 0x7f8ff406d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): picked mon.noname-a con 0x7f8ff40c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff415a5e0 0x7f8ff406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): picked mon.noname-c con 0x7f8ff415a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.382 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4150520 con 0x7f8ff40c99a0 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40c8a10 con 0x7f8ff406ff20 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41506a0 con 0x7f8ff415a5e0 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff9fe3640 10 monclient(hunting): authenticate will time out at 1396.100226s 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff415a5e0 0x7f8ff406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff2ffd640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff40c99a0 0x7f8ff406d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff37fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff406ff20 0x7f8ff40bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff415a5e0 0x7f8ff406d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43270/0 (socket says 10.20.193.6:43270) 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 1 -- 10.20.193.6:0/3691335572 learned_addr learned my addr 10.20.193.6:0/3691335572 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:41.383 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 10 monclient(hunting): get_auth_request con 0x7f8ff415a5e0 auth_method 0 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff2ffd640 10 monclient(hunting): get_auth_request con 0x7f8ff40c99a0 auth_method 0 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff2ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff2ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff2ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff37fe640 10 monclient(hunting): get_auth_request con 0x7f8ff406ff20 auth_method 0 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.384 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.373+0000 7f8ff3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.374+0000 7f8ff2ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.374+0000 7f8ff2ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.374+0000 7f8ff2ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.374+0000 7f8ff37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.374+0000 7f8ff37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.374+0000 7f8ff37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 10 monclient(hunting): handle_auth_done global_id 5390 payload 824 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 1 -- 10.20.193.6:0/3691335572 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff406ff20 msgr2=0x7f8ff40bcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff406ff20 0x7f8ff40bcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 1 -- 10.20.193.6:0/3691335572 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff40c99a0 msgr2=0x7f8ff406d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff40c99a0 0x7f8ff406d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.385 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 1 monclient: found mon.noname-c 2026-02-22T23:28:41.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.386 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.375+0000 7f8ff3fff640 1 -- 10.20.193.6:0/3691335572 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff406de00 con 0x7f8ff415a5e0 2026-02-22T23:28:41.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.376+0000 7f8ff3fff640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff415a5e0 0x7f8ff406d7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800caa0 tx=0x7f8fe800cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.376+0000 7f8fcffff640 10 client.?.objecter ms_handle_connect 0x7f8ff415a5e0 2026-02-22T23:28:41.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.376+0000 7f8fcffff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:41.388 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 1 -- 10.20.193.6:0/3691335572 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800ac40 con 0x7f8ff415a5e0 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: dump: 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.389 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 1 -- 10.20.193.6:0/3691335572 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe800ade0 con 0x7f8ff415a5e0 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 5 monclient: authenticate success, global_id 5390 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 10 monclient: _renew_subs 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff406e0e0 con 0x7f8ff415a5e0 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 1 -- 10.20.193.6:0/3691335572 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8012600 con 0x7f8ff415a5e0 2026-02-22T23:28:41.390 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8fcffff640 10 monclient: dump: 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.391 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff1ffb640 4 set_mon_vals no callback set 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 10 client.5390.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 10 monclient: _renew_subs 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff406e6a0 con 0x7f8ff415a5e0 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.377+0000 7f8ff9fe3640 1 librados: init done 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.378+0000 7f8ff9fe3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.378+0000 7f8ff9fe3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.392 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.378+0000 7f8ff9fe3640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.393 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.378+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff40c8ef0 con 0x7f8ff415a5e0 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.389+0000 7f8fcffff640 1 -- 10.20.193.6:0/3691335572 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f8fe80127a0 con 0x7f8ff415a5e0 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.389+0000 7f8fcffff640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc4078e50 0x7f8fc407b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.389+0000 7f8ff37fe640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc4078e50 0x7f8fc407b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.389+0000 7f8ff37fe640 10 monclient: get_auth_request con 0x7f8fc4078e50 auth_method 0 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.390+0000 7f8fcffff640 1 -- 10.20.193.6:0/3691335572 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f8fe805e860 con 0x7f8ff415a5e0 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.390+0000 7f8fcffff640 10 client.5390.objecter ms_dispatch2 0x7f8ff4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.390+0000 7f8fcffff640 3 client.5390.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.390+0000 7f8fcffff640 3 client.5390.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:41.394 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.390+0000 7f8fcffff640 20 client.5390.objecter dump_active .. 0 homeless 2026-02-22T23:28:41.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.390+0000 7f8ff37fe640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc4078e50 0x7f8fc407b310 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0002770 tx=0x7f8fe0007ef0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.395 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.390+0000 7f8fcffff640 10 client.5390.objecter ms_handle_connect 0x7f8fc4078e50 2026-02-22T23:28:41.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.487+0000 7f8fcffff640 1 -- 10.20.193.6:0/3691335572 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f8fe8062820 con 0x7f8ff415a5e0 2026-02-22T23:28:41.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.487+0000 7f8fcffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.488 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.487+0000 7f8fcffff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.573+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f8ff40c48e0 con 0x7f8fc4078e50 2026-02-22T23:28:41.575 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.573+0000 7f8fcffff640 1 -- 10.20.193.6:0/3691335572 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8ff40c48e0 con 0x7f8fc4078e50 2026-02-22T23:28:41.575 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 3 to deep-scrub 2026-02-22T23:28:41.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 10 librados: watch_flush enter 2026-02-22T23:28:41.576 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 10 librados: watch_flush exit 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 20 client.5390.objecter shutdown clearing up homeless session... 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 10 client.5390.objecter successfully canceled tick 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc4078e50 msgr2=0x7f8fc407b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc4078e50 0x7f8fc407b310 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0002770 tx=0x7f8fe0007ef0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 10 monclient: shutdown 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff415a5e0 msgr2=0x7f8ff406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff415a5e0 0x7f8ff406d7c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800caa0 tx=0x7f8fe800cf70 comp rx=0 tx=0).stop 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 shutdown_connections 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f8fc4078e50 0x7f8fc407b310 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.577 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f8ff415a5e0 0x7f8ff406d7c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f8ff40c99a0 0x7f8ff406d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 --2- 10.20.193.6:0/3691335572 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f8ff406ff20 0x7f8ff40bcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 >> 10.20.193.6:0/3691335572 conn(0x7f8ff40cd780 msgr2=0x7f8ff414e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 shutdown_connections 2026-02-22T23:28:41.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 -- 10.20.193.6:0/3691335572 wait complete. 2026-02-22T23:28:41.578 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.574+0000 7f8ff9fe3640 1 librados: shutdown 2026-02-22T23:28:41.584 INFO:tasks.ceph:Scrubbing osd.7 2026-02-22T23:28:41.585 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:41.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.653+0000 7f97da674640 10 monclient: get_monmap_and_config 2026-02-22T23:28:41.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.653+0000 7f97da674640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.653+0000 7f97da674640 10 monclient: monmap: 2026-02-22T23:28:41.655 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.655 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.655 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:41.654489+0000 2026-02-22T23:28:41.655 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:41.654489+0000 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:41.656 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 1 Processor -- start 2026-02-22T23:28:41.659 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 1 -- start start 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient: init 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient: _add_conns ranks=[5,4,1,3,2,0] 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f97d41503e0 0x7f97d4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient(hunting): picked mon.noname-c con 0x7f97d41503e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f97d415a070 0x7f97d406fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient(hunting): picked mon.noname-b con 0x7f97d415a070 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f97d40702f0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d415a570 con 0x7f97d40702f0 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f97d40c7d70 con 0x7f97d415a070 2026-02-22T23:28:41.660 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7f97d40c8610 con 0x7f97d41503e0 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.654+0000 7f97da674640 10 monclient(hunting): authenticate will time out at 1396.381868s 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f97d41503e0 0x7f97d4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d37fe640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f97d415a070 0x7f97d406fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7f97d41503e0 0x7f97d4159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43280/0 (socket says 10.20.193.6:43280) 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 1 -- 10.20.193.6:0/1520608209 learned_addr learned my addr 10.20.193.6:0/1520608209 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 10 monclient(hunting): get_auth_request con 0x7f97d41503e0 auth_method 0 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.661 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d37fe640 10 monclient(hunting): get_auth_request con 0x7f97d415a070 auth_method 0 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.655+0000 7f97d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 10 monclient(hunting): handle_auth_done global_id 5393 payload 274 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 1 -- 10.20.193.6:0/1520608209 >> v2:10.20.193.40:3300/0 conn(0x7f97d415a070 msgr2=0x7f97d406fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 1 --2- 10.20.193.6:0/1520608209 >> v2:10.20.193.40:3300/0 conn(0x7f97d415a070 0x7f97d406fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.662 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 1 -- 10.20.193.6:0/1520608209 >> v1:10.20.193.40:6789/0 conn(0x7f97d40702f0 legacy=0x7f97d40c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 1 monclient: found mon.noname-c 2026-02-22T23:28:41.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.656+0000 7f97d3fff640 1 -- 10.20.193.6:0/1520608209 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d415a850 con 0x7f97d41503e0 2026-02-22T23:28:41.663 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.657+0000 7f97d3fff640 1 --2- 10.20.193.6:0/1520608209 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4159b30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f97d40ce490 tx=0x7f97c002ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5bb39b1d6bf646 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 1 -- 10.20.193.6:0/1520608209 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c003c070 con 0x7f97d41503e0 2026-02-22T23:28:41.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.669 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: dump: 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.670 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 1 -- 10.20.193.6:0/1520608209 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c002fc70 con 0x7f97d41503e0 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 5 monclient: authenticate success, global_id 5393 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 1 -- 10.20.193.6:0/1520608209 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0037660 con 0x7f97d41503e0 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97d2ffd640 10 monclient: dump: 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.671 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 4 set_mon_vals no callback set 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 10 monclient: shutdown 2026-02-22T23:28:41.672 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 -- 10.20.193.6:0/1520608209 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 msgr2=0x7f97d4159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 --2- 10.20.193.6:0/1520608209 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4159b30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f97d40ce490 tx=0x7f97c002ef60 comp rx=0 tx=0).stop 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 -- 10.20.193.6:0/1520608209 shutdown_connections 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 --2- 10.20.193.6:0/1520608209 >> v2:10.20.193.40:3300/0 conn(0x7f97d415a070 0x7f97d406fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 --2- 10.20.193.6:0/1520608209 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4159b30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 -- 10.20.193.6:0/1520608209 >> 10.20.193.6:0/1520608209 conn(0x7f97d40cd600 msgr2=0x7f97d40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 -- 10.20.193.6:0/1520608209 shutdown_connections 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.659+0000 7f97da674640 1 -- 10.20.193.6:0/1520608209 wait complete. 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 10 monclient: monmap: 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.673 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 1 librados: starting msgr at 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 1 librados: starting objecter 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 1 Processor -- start 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 1 -- start start 2026-02-22T23:28:41.674 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 1 librados: setting wanted keys 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 1 librados: calling monclient init 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.660+0000 7f97da674640 10 monclient: init 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f97d40702f0 0x7f97d41f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): picked mon.noname-a con 0x7f97d40702f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): picked mon.noname-c con 0x7f97d41503e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f97d415a070 0x7f97d40c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): picked mon.noname-b con 0x7f97d415a070 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.675 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f97d40c82a0 con 0x7f97d40702f0 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f97d4070760 con 0x7f97d415a070 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f97d41507c0 con 0x7f97d41503e0 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97da674640 10 monclient(hunting): authenticate will time out at 1396.388413s 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d37fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d8bea640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f97d415a070 0x7f97d40c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d37fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43294/0 (socket says 10.20.193.6:43294) 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d37fe640 1 -- 10.20.193.6:0/3064548209 learned_addr learned my addr 10.20.193.6:0/3064548209 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d37fe640 10 monclient(hunting): get_auth_request con 0x7f97d41503e0 auth_method 0 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d37fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.676 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d37fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d37fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.661+0000 7f97d3fff640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f97d40702f0 0x7f97d41f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d8bea640 10 monclient(hunting): get_auth_request con 0x7f97d415a070 auth_method 0 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d8bea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d8bea640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d8bea640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d37fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d37fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d37fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d3fff640 10 monclient(hunting): get_auth_request con 0x7f97d40702f0 auth_method 0 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d3fff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d3fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.677 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d3fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d8bea640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d8bea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d8bea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d3fff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d3fff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.662+0000 7f97d3fff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 10 monclient(hunting): handle_auth_done global_id 5396 payload 824 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 1 -- 10.20.193.6:0/3064548209 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f97d415a070 msgr2=0x7f97d40c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f97d415a070 0x7f97d40c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 1 -- 10.20.193.6:0/3064548209 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f97d40702f0 msgr2=0x7f97d41f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f97d40702f0 0x7f97d41f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.678 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 1 monclient: found mon.noname-c 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.663+0000 7f97d37fe640 1 -- 10.20.193.6:0/3064548209 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d4151240 con 0x7f97d41503e0 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.664+0000 7f97d37fe640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4153730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e890 tx=0x7f97c400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.664+0000 7f97d0ff9640 10 client.?.objecter ms_handle_connect 0x7f97d41503e0 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.664+0000 7f97d0ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4017070 con 0x7f97d41503e0 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97d0ff9640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97d0ff9640 10 monclient: dump: 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.679 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97d0ff9640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97d0ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 5 monclient: authenticate success, global_id 5396 2026-02-22T23:28:41.680 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 10 monclient: _renew_subs 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d4150f60 con 0x7f97d41503e0 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 10 client.5396.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 10 monclient: _renew_subs 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d4152c50 con 0x7f97d41503e0 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.665+0000 7f97da674640 1 librados: init done 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c400c940 con 0x7f97d41503e0 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97d0ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c4021060 con 0x7f97d41503e0 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:41.681 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97d27fc640 4 set_mon_vals no callback set 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97d0ff9640 10 monclient: dump: 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97da674640 10 client.5396.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.682 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97da674640 15 client.5396.objecter _session_command_op_assign -1 1 2026-02-22T23:28:41.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97da674640 20 client.5396.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:41.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97da674640 10 client.5396.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:41.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97da674640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:28:41.683 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.666+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f97d4159b30 con 0x7f97d41503e0 2026-02-22T23:28:41.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.677+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f97c4013470 con 0x7f97d41503e0 2026-02-22T23:28:41.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.678+0000 7f97d0ff9640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f97a4078e00 0x7f97a407b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.678+0000 7f97d3fff640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f97a4078e00 0x7f97a407b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.678+0000 7f97d3fff640 10 monclient: get_auth_request con 0x7f97a4078e00 auth_method 0 2026-02-22T23:28:41.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.679+0000 7f97d3fff640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f97a4078e00 0x7f97a407b2c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f97d406f550 tx=0x7f97c0039040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.685 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.679+0000 7f97d0ff9640 10 client.5396.objecter ms_handle_connect 0x7f97a4078e00 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f97c4097e10 con 0x7f97d41503e0 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 10 client.5396.objecter ms_dispatch2 0x7f97d4000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 3 client.5396.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 3 client.5396.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 10 client.5396.objecter checking command 1 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7f97a40812e0 0x7f97a4083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 20 client.5396.objecter _get_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 20 client.5396.objecter put_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 15 client.5396.objecter _session_command_op_remove -1 1 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 20 client.5396.objecter _get_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 20 client.5396.objecter get_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.686 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 15 client.5396.objecter _session_command_op_assign 7 1 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 20 client.5396.objecter put_session s=0x7f97a4080c10 osd=7 4 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 10 client.5396.objecter _send_command 1 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 --> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97c401fe70 con 0x7f97a40812e0 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 20 client.5396.objecter dump_active .. 0 homeless 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97c405e8d0 con 0x7f97d41503e0 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d0ff9640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.680+0000 7f97d8bea640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7f97a40812e0 0x7f97a4083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.681+0000 7f97d8bea640 10 monclient: get_auth_request con 0x7f97a40812e0 auth_method 0 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.682+0000 7f97d8bea640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7f97a40812e0 0x7f97a4083700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.687 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.682+0000 7f97d0ff9640 10 client.5396.objecter ms_handle_connect 0x7f97a40812e0 2026-02-22T23:28:41.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.693+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== osd.7 v2:10.20.193.189:6810/980553521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7f97c401fe70 con 0x7f97a40812e0 2026-02-22T23:28:41.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.693+0000 7f97d0ff9640 10 client.5396.objecter ms_dispatch2 0x7f97d4000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:41.694 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.693+0000 7f97d0ff9640 10 client.5396.objecter _finish_command 1 = system:0 2026-02-22T23:28:41.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.693+0000 7f97d0ff9640 20 client.5396.objecter put_session s=0x7f97a4080c10 osd=7 4 2026-02-22T23:28:41.695 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.693+0000 7f97d0ff9640 15 client.5396.objecter _session_command_op_remove 7 1 2026-02-22T23:28:41.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 10 client.5396.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:41.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 15 client.5396.objecter _session_command_op_assign -1 2 2026-02-22T23:28:41.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 20 client.5396.objecter _get_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.701 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 20 client.5396.objecter put_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 20 client.5396.objecter _get_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 15 client.5396.objecter _session_command_op_remove -1 2 2026-02-22T23:28:41.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 20 client.5396.objecter get_session s=0x7f97a4080c10 osd=7 3 2026-02-22T23:28:41.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 15 client.5396.objecter _session_command_op_assign 7 2 2026-02-22T23:28:41.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 20 client.5396.objecter put_session s=0x7f97a4080c10 osd=7 4 2026-02-22T23:28:41.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 10 client.5396.objecter _send_command 2 2026-02-22T23:28:41.702 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.700+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 --> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f97d414d5b0 con 0x7f97a40812e0 2026-02-22T23:28:41.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97d0ff9640 1 -- 10.20.193.6:0/3064548209 <== osd.7 v2:10.20.193.189:6810/980553521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f97d414d5b0 con 0x7f97a40812e0 2026-02-22T23:28:41.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97d0ff9640 10 client.5396.objecter ms_dispatch2 0x7f97d4000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:41.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97d0ff9640 10 client.5396.objecter _finish_command 2 = system:0 2026-02-22T23:28:41.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97d0ff9640 20 client.5396.objecter put_session s=0x7f97a4080c10 osd=7 4 2026-02-22T23:28:41.708 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97d0ff9640 15 client.5396.objecter _session_command_op_remove 7 2 2026-02-22T23:28:41.708 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:41.709 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:41.709 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:41.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 10 librados: watch_flush enter 2026-02-22T23:28:41.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 10 librados: watch_flush exit 2026-02-22T23:28:41.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 10 client.5396.objecter close_session for osd.7 2026-02-22T23:28:41.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7f97a40812e0 msgr2=0x7f97a4083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6810/980553521,v1:10.20.193.189:6811/980553521] conn(0x7f97a40812e0 0x7f97a4083700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 20 client.5396.objecter put_session s=0x7f97a4080c10 osd=7 1 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 20 client.5396.objecter shutdown clearing up homeless session... 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 10 client.5396.objecter successfully canceled tick 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f97a4078e00 msgr2=0x7f97a407b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f97a4078e00 0x7f97a407b2c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f97d406f550 tx=0x7f97c0039040 comp rx=0 tx=0).stop 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 10 monclient: shutdown 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 msgr2=0x7f97d4153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.706+0000 7f97da674640 1 --2- 10.20.193.6:0/3064548209 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f97d41503e0 0x7f97d4153730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f97c400e890 tx=0x7f97c400ed60 comp rx=0 tx=0).stop 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.707+0000 7f97d8bea640 1 -- 10.20.193.6:0/3064548209 reap_dead start 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.707+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 shutdown_connections 2026-02-22T23:28:41.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.707+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 >> 10.20.193.6:0/3064548209 conn(0x7f97d40cd600 msgr2=0x7f97d414e070 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.707+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 shutdown_connections 2026-02-22T23:28:41.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.707+0000 7f97da674640 1 -- 10.20.193.6:0/3064548209 wait complete. 2026-02-22T23:28:41.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.707+0000 7f97da674640 1 librados: shutdown 2026-02-22T23:28:41.716 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2026-02-22T23:28:41.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.781+0000 7fb250838640 10 monclient: get_monmap_and_config 2026-02-22T23:28:41.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.781+0000 7fb250838640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.783 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.781+0000 7fb250838640 10 monclient: monmap: 2026-02-22T23:28:41.783 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.783 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.783 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:41.782453+0000 2026-02-22T23:28:41.783 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:41.782453+0000 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:41.784 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 1 Processor -- start 2026-02-22T23:28:41.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 1 -- start start 2026-02-22T23:28:41.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient: init 2026-02-22T23:28:41.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.787 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient: _add_conns ranks=[0,4,2,1,5,3] 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb24806ff20 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fb2480c99a0 0x7fb248070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient(hunting): picked mon.noname-b con 0x7fb2480c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb24815a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2480c7e90 con 0x7fb24806ff20 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2480c8170 con 0x7fb24815a5e0 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fb248159a40 con 0x7fb2480c99a0 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.782+0000 7fb250838640 10 monclient(hunting): authenticate will time out at 1396.509846s 2026-02-22T23:28:41.788 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24e5ad640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fb24806ff20 0x7fb248159ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52278/0 (socket says 10.20.193.6:52278) 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24e5ad640 1 -- 10.20.193.6:0/595878685 learned_addr learned my addr 10.20.193.6:0/595878685 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 1 --2- 10.20.193.6:0/595878685 >> v2:10.20.193.40:3300/0 conn(0x7fb2480c99a0 0x7fb248070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 10 monclient(hunting): get_auth_request con 0x7fb2480c99a0 auth_method 0 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.783+0000 7fb24ddac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24d5ab640 1 -- 10.20.193.6:0/595878685 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 544514500 0 0) 0x7fb2480c7e90 con 0x7fb24806ff20 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24d5ab640 10 monclient(hunting): _negotiate 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24d5ab640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.789 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24d5ab640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24d5ab640 10 monclient(hunting): my global_id is 5277 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24d5ab640 1 -- 10.20.193.6:0/595878685 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb238003620 con 0x7fb24806ff20 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 10 monclient(hunting): handle_auth_done global_id 5251 payload 274 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 1 -- 10.20.193.6:0/595878685 >> v1:10.20.193.80:6789/0 conn(0x7fb24815a5e0 legacy=0x7fb24814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 1 -- 10.20.193.6:0/595878685 >> v1:10.20.193.6:6789/0 conn(0x7fb24806ff20 legacy=0x7fb248159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 1 monclient: found mon.noname-b 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.784+0000 7fb24ddac640 1 -- 10.20.193.6:0/595878685 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2480c8450 con 0x7fb2480c99a0 2026-02-22T23:28:41.790 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.785+0000 7fb24ddac640 1 --2- 10.20.193.6:0/595878685 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2480c99a0 0x7fb248070300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2340098d0 tx=0x7fb23402eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a262a061e896f470 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 1 -- 10.20.193.6:0/595878685 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23403c070 con 0x7fb2480c99a0 2026-02-22T23:28:41.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.797 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: dump: 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.798 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 1 -- 10.20.193.6:0/595878685 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23402fba0 con 0x7fb2480c99a0 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 1 -- 10.20.193.6:0/595878685 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234037440 con 0x7fb2480c99a0 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 5 monclient: authenticate success, global_id 5251 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb24d5ab640 10 monclient: dump: 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.799 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 4 set_mon_vals no callback set 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 10 monclient: shutdown 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 1 -- 10.20.193.6:0/595878685 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2480c99a0 msgr2=0x7fb248070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.800 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 1 --2- 10.20.193.6:0/595878685 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2480c99a0 0x7fb248070300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb2340098d0 tx=0x7fb23402eed0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 1 -- 10.20.193.6:0/595878685 shutdown_connections 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 1 --2- 10.20.193.6:0/595878685 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb2480c99a0 0x7fb248070300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 1 -- 10.20.193.6:0/595878685 >> 10.20.193.6:0/595878685 conn(0x7fb2480cd780 msgr2=0x7fb2480cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 1 -- 10.20.193.6:0/595878685 shutdown_connections 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.787+0000 7fb250838640 1 -- 10.20.193.6:0/595878685 wait complete. 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient: build_initial_monmap 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient: monmap: 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:41.801 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 librados: starting msgr at 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 librados: starting objecter 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 Processor -- start 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 -- start start 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 librados: setting wanted keys 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 librados: calling monclient init 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient: init 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:41.802 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb24806ff20 0x7fb2480c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): picked mon.noname-b con 0x7fb24806ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb2480c99a0 0x7fb24806e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): picked mon.noname-a con 0x7fb2480c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb24815a5e0 0x7fb24814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): picked mon.noname-c con 0x7fb24815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb248150520 con 0x7fb2480c99a0 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb2480c8a10 con 0x7fb24806ff20 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:41.803 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481506a0 con 0x7fb24815a5e0 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.788+0000 7fb250838640 10 monclient(hunting): authenticate will time out at 1396.515901s 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb2480c99a0 0x7fb24806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb24806ff20 0x7fb2480c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb24815a5e0 0x7fb24814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb2480c99a0 0x7fb24806e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37116/0 (socket says 10.20.193.6:37116) 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 1 -- 10.20.193.6:0/504542229 learned_addr learned my addr 10.20.193.6:0/504542229 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 10 monclient(hunting): get_auth_request con 0x7fb2480c99a0 auth_method 0 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.804 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 10 monclient(hunting): get_auth_request con 0x7fb24806ff20 auth_method 0 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 10 monclient(hunting): get_auth_request con 0x7fb24815a5e0 auth_method 0 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24e5ad640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.805 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24ddac640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.789+0000 7fb24edae640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.790+0000 7fb24ddac640 10 monclient(hunting): handle_auth_done global_id 5280 payload 824 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.790+0000 7fb24ddac640 1 -- 10.20.193.6:0/504542229 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb24815a5e0 msgr2=0x7fb24814bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.790+0000 7fb24ddac640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb24815a5e0 0x7fb24814bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 1 -- 10.20.193.6:0/504542229 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb24806ff20 msgr2=0x7fb2480c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb24806ff20 0x7fb2480c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 10 monclient: _finish_hunting 0 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 1 monclient: found mon.noname-a 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:41.806 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 1 -- 10.20.193.6:0/504542229 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2480c12e0 con 0x7fb2480c99a0 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24e5ad640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb24806ff20 0x7fb2480c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb24ddac640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb2480c99a0 0x7fb24806e540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb23403a040 tx=0x7fb2340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb232ffd640 10 client.?.objecter ms_handle_connect 0x7fb2480c99a0 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.791+0000 7fb232ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 1 -- 10.20.193.6:0/504542229 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234046070 con 0x7fb2480c99a0 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: dump: 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.807 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 1 -- 10.20.193.6:0/504542229 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb234004230 con 0x7fb2480c99a0 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 5 monclient: authenticate success, global_id 5280 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:41.808 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 1 -- 10.20.193.6:0/504542229 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23403c040 con 0x7fb2480c99a0 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 10 monclient: _renew_subs 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb24cdaa640 4 set_mon_vals no callback set 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2480c15c0 con 0x7fb2480c99a0 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb232ffd640 10 monclient: dump: 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:41.809 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 10 client.5280.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 10 monclient: _renew_subs 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2480c1b80 con 0x7fb2480c99a0 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.793+0000 7fb250838640 1 librados: init done 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.794+0000 7fb250838640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.794+0000 7fb250838640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.794+0000 7fb250838640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:41.810 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.794+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb248070300 con 0x7fb2480c99a0 2026-02-22T23:28:41.811 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.805+0000 7fb232ffd640 1 -- 10.20.193.6:0/504542229 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb234003710 con 0x7fb2480c99a0 2026-02-22T23:28:41.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.806+0000 7fb232ffd640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb218078e50 0x7fb21807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:41.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.806+0000 7fb24e5ad640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb218078e50 0x7fb21807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:41.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.806+0000 7fb24e5ad640 10 monclient: get_auth_request con 0x7fb218078e50 auth_method 0 2026-02-22T23:28:41.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.807+0000 7fb24e5ad640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb218078e50 0x7fb21807b310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb240005980 tx=0x7fb240007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:41.812 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.807+0000 7fb232ffd640 10 client.5280.objecter ms_handle_connect 0x7fb218078e50 2026-02-22T23:28:41.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.843+0000 7fb232ffd640 1 -- 10.20.193.6:0/504542229 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fb23402fa60 con 0x7fb2480c99a0 2026-02-22T23:28:41.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.843+0000 7fb232ffd640 10 client.5280.objecter ms_dispatch2 0x7fb248000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:41.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.843+0000 7fb232ffd640 3 client.5280.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:41.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.843+0000 7fb232ffd640 3 client.5280.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:41.845 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.844+0000 7fb232ffd640 20 client.5280.objecter dump_active .. 0 homeless 2026-02-22T23:28:41.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.905+0000 7fb232ffd640 1 -- 10.20.193.6:0/504542229 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb234051080 con 0x7fb2480c99a0 2026-02-22T23:28:41.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.905+0000 7fb232ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:41.907 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.905+0000 7fb232ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:41.991 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.990+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb2480bd740 con 0x7fb218078e50 2026-02-22T23:28:41.992 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.990+0000 7fb232ffd640 1 -- 10.20.193.6:0/504542229 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb2480bd740 con 0x7fb218078e50 2026-02-22T23:28:41.992 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 7 to deep-scrub 2026-02-22T23:28:41.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 10 librados: watch_flush enter 2026-02-22T23:28:41.993 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 10 librados: watch_flush exit 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 20 client.5280.objecter shutdown clearing up homeless session... 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 10 client.5280.objecter successfully canceled tick 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb218078e50 msgr2=0x7fb21807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb218078e50 0x7fb21807b310 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb240005980 tx=0x7fb240007dd0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 10 monclient: shutdown 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb2480c99a0 msgr2=0x7fb24806e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb2480c99a0 0x7fb24806e540 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb23403a040 tx=0x7fb2340047c0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 shutdown_connections 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb218078e50 0x7fb21807b310 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.994 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb24815a5e0 0x7fb24814bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb2480c99a0 0x7fb24806e540 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 --2- 10.20.193.6:0/504542229 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb24806ff20 0x7fb2480c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 >> 10.20.193.6:0/504542229 conn(0x7fb2480cd780 msgr2=0x7fb2480cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 shutdown_connections 2026-02-22T23:28:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 -- 10.20.193.6:0/504542229 wait complete. 2026-02-22T23:28:41.995 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:41.991+0000 7fb250838640 1 librados: shutdown 2026-02-22T23:28:42.002 INFO:tasks.ceph:Scrubbing osd.11 2026-02-22T23:28:42.002 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:42.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.070+0000 7fd66b32c640 10 monclient: get_monmap_and_config 2026-02-22T23:28:42.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.070+0000 7fd66b32c640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.072 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.070+0000 7fd66b32c640 10 monclient: monmap: 2026-02-22T23:28:42.072 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.072 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.072 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:42.071448+0000 2026-02-22T23:28:42.072 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:42.071448+0000 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:42.073 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 1 Processor -- start 2026-02-22T23:28:42.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 1 -- start start 2026-02-22T23:28:42.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient: init 2026-02-22T23:28:42.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient: _add_conns ranks=[5,2,1,4,3,0] 2026-02-22T23:28:42.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fd6641503e0 0x7fd664159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.076 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient(hunting): picked mon.noname-c con 0x7fd6641503e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fd66415a070 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fd6640704c0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66415a570 con 0x7fd6640704c0 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66415a850 con 0x7fd66415a070 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd6640c7d70 con 0x7fd6641503e0 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.071+0000 7fd66b32c640 10 monclient(hunting): authenticate will time out at 1396.798850s 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.072+0000 7fd6698a2640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fd6640704c0 0x7fd6640c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38388/0 (socket says 10.20.193.6:38388) 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.072+0000 7fd6698a2640 1 -- 10.20.193.6:0/4263240522 learned_addr learned my addr 10.20.193.6:0/4263240522 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:42.077 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.072+0000 7fd6690a1640 1 --2- 10.20.193.6:0/4263240522 >> v2:10.20.193.80:3300/0 conn(0x7fd6641503e0 0x7fd664159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.073+0000 7fd6690a1640 10 monclient(hunting): get_auth_request con 0x7fd6641503e0 auth_method 0 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.073+0000 7fd6690a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.073+0000 7fd6690a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.073+0000 7fd6690a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.073+0000 7fd6690a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.073+0000 7fd6690a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.073+0000 7fd6690a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd653fff640 1 -- 10.20.193.6:0/4263240522 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214677029 0 0) 0x7fd66415a570 con 0x7fd6640704c0 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd653fff640 10 monclient(hunting): _negotiate 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd653fff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.078 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd653fff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd653fff640 10 monclient(hunting): my global_id is 5257 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd653fff640 1 -- 10.20.193.6:0/4263240522 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd64c003620 con 0x7fd6640704c0 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 10 monclient(hunting): handle_auth_done global_id 5405 payload 274 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 1 -- 10.20.193.6:0/4263240522 >> v1:10.20.193.80:6789/0 conn(0x7fd66415a070 legacy=0x7fd66406fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 1 -- 10.20.193.6:0/4263240522 >> v1:10.20.193.40:6789/0 conn(0x7fd6640704c0 legacy=0x7fd6640c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 1 monclient: found mon.noname-c 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:28:42.079 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.074+0000 7fd6690a1640 1 -- 10.20.193.6:0/4263240522 --> v2:10.20.193.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6640c7fc0 con 0x7fd6641503e0 2026-02-22T23:28:42.084 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.075+0000 7fd6690a1640 1 --2- 10.20.193.6:0/4263240522 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd6641503e0 0x7fd664159b30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd6540098a0 tx=0x7fd65402ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57e7805adfc8b18 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 1 -- 10.20.193.6:0/4263240522 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65403c070 con 0x7fd6641503e0 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: dump: 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.085 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 1 -- 10.20.193.6:0/4263240522 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65402fc10 con 0x7fd6641503e0 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 5 monclient: authenticate success, global_id 5405 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 1 -- 10.20.193.6:0/4263240522 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654037440 con 0x7fd6641503e0 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.086 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd653fff640 10 monclient: dump: 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.087 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 4 set_mon_vals no callback set 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 10 monclient: shutdown 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 1 -- 10.20.193.6:0/4263240522 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd6641503e0 msgr2=0x7fd664159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 1 --2- 10.20.193.6:0/4263240522 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd6641503e0 0x7fd664159b30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd6540098a0 tx=0x7fd65402ef00 comp rx=0 tx=0).stop 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 1 -- 10.20.193.6:0/4263240522 shutdown_connections 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 1 --2- 10.20.193.6:0/4263240522 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd6641503e0 0x7fd664159b30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 1 -- 10.20.193.6:0/4263240522 >> 10.20.193.6:0/4263240522 conn(0x7fd6640cd600 msgr2=0x7fd6640cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 1 -- 10.20.193.6:0/4263240522 shutdown_connections 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.078+0000 7fd66b32c640 1 -- 10.20.193.6:0/4263240522 wait complete. 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.088 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient: monmap: 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 librados: starting msgr at 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 librados: starting objecter 2026-02-22T23:28:42.089 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 Processor -- start 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 -- start start 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 librados: setting wanted keys 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 librados: calling monclient init 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient: init 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd6640704c0 0x7fd6641447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): picked mon.noname-a con 0x7fd6640704c0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd6641503e0 0x7fd66415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): picked mon.noname-b con 0x7fd6641503e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd66415a070 0x7fd66415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): picked mon.noname-c con 0x7fd66415a070 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:42.090 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641507c0 con 0x7fd6640704c0 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fd6640c8610 con 0x7fd6641503e0 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd664150940 con 0x7fd66415a070 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.079+0000 7fd66b32c640 10 monclient(hunting): authenticate will time out at 1396.806911s 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd6640704c0 0x7fd6641447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd66415a070 0x7fd66415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd6641503e0 0x7fd66415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd6640704c0 0x7fd6641447b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37128/0 (socket says 10.20.193.6:37128) 2026-02-22T23:28:42.091 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 1 -- 10.20.193.6:0/2573351888 learned_addr learned my addr 10.20.193.6:0/2573351888 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 10 monclient(hunting): get_auth_request con 0x7fd6640704c0 auth_method 0 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 10 monclient(hunting): get_auth_request con 0x7fd66415a070 auth_method 0 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 10 monclient(hunting): get_auth_request con 0x7fd6641503e0 auth_method 0 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.092 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6690a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6688a0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.080+0000 7fd6698a2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 10 monclient(hunting): handle_auth_done global_id 5260 payload 824 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 1 -- 10.20.193.6:0/2573351888 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd66415a070 msgr2=0x7fd66415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd66415a070 0x7fd66415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 1 -- 10.20.193.6:0/2573351888 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd6640704c0 msgr2=0x7fd6641447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.093 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd6640704c0 0x7fd6641447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 1 monclient: found mon.noname-b 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6688a0640 1 -- 10.20.193.6:0/2573351888 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66414ca50 con 0x7fd6641503e0 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6698a2640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fd66415a070 0x7fd66415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.082+0000 7fd6690a1640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fd6640704c0 0x7fd6641447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.083+0000 7fd6688a0640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd6641503e0 0x7fd66415b060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c430 tx=0x7fd65800c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.083+0000 7fd651ffb640 10 client.?.objecter ms_handle_connect 0x7fd6641503e0 2026-02-22T23:28:42.094 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.083+0000 7fd651ffb640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:42.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658016020 con 0x7fd6641503e0 2026-02-22T23:28:42.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:42.096 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: dump: 2026-02-22T23:28:42.096 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.096 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.097 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 5 monclient: authenticate success, global_id 5260 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd658005310 con 0x7fd6641503e0 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 monclient: _renew_subs 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66414cd30 con 0x7fd6641503e0 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6580068d0 con 0x7fd6641503e0 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:42.098 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd651ffb640 10 monclient: dump: 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd6537fe640 4 set_mon_vals no callback set 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 client.5260.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:42.099 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 monclient: _renew_subs 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66414d2f0 con 0x7fd6641503e0 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 1 librados: init done 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 client.5260.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 15 client.5260.objecter _session_command_op_assign -1 1 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 20 client.5260.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 client.5260.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.100 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.085+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd664159b30 con 0x7fd6641503e0 2026-02-22T23:28:42.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.097+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fd658006a70 con 0x7fd6641503e0 2026-02-22T23:28:42.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.098+0000 7fd651ffb640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd634078e30 0x7fd63407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.098+0000 7fd6690a1640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd634078e30 0x7fd63407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.102 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.098+0000 7fd6690a1640 10 monclient: get_auth_request con 0x7fd634078e30 auth_method 0 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.098+0000 7fd651ffb640 10 client.5260.objecter ms_handle_connect 0x7fd634078e30 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.098+0000 7fd6690a1640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd634078e30 0x7fd63407b2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd654016ed0 tx=0x7fd654033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fd658006d90 con 0x7fd6641503e0 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 10 client.5260.objecter ms_dispatch2 0x7fd664000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 3 client.5260.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 3 client.5260.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 10 client.5260.objecter checking command 1 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fd634081310 0x7fd634083730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 20 client.5260.objecter _get_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 20 client.5260.objecter put_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 15 client.5260.objecter _session_command_op_remove -1 1 2026-02-22T23:28:42.103 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 20 client.5260.objecter _get_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 20 client.5260.objecter get_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 15 client.5260.objecter _session_command_op_assign 11 1 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 20 client.5260.objecter put_session s=0x7fd634080c40 osd=11 4 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 10 client.5260.objecter _send_command 1 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 --> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd65801fe50 con 0x7fd634081310 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.099+0000 7fd651ffb640 20 client.5260.objecter dump_active .. 0 homeless 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.100+0000 7fd6698a2640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fd634081310 0x7fd634083730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.100+0000 7fd6698a2640 10 monclient: get_auth_request con 0x7fd634081310 auth_method 0 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.100+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd65805e320 con 0x7fd6641503e0 2026-02-22T23:28:42.104 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.100+0000 7fd651ffb640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:42.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.102+0000 7fd651ffb640 10 client.5260.objecter ms_handle_connect 0x7fd634081310 2026-02-22T23:28:42.105 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.102+0000 7fd6698a2640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fd634081310 0x7fd634083730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.112+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== osd.11 v2:10.20.193.189:6802/2070530833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fd65801fe50 con 0x7fd634081310 2026-02-22T23:28:42.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.112+0000 7fd651ffb640 10 client.5260.objecter ms_dispatch2 0x7fd664000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:42.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.112+0000 7fd651ffb640 10 client.5260.objecter _finish_command 1 = system:0 2026-02-22T23:28:42.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.112+0000 7fd651ffb640 20 client.5260.objecter put_session s=0x7fd634080c40 osd=11 4 2026-02-22T23:28:42.113 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.112+0000 7fd651ffb640 15 client.5260.objecter _session_command_op_remove 11 1 2026-02-22T23:28:42.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 10 client.5260.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:42.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 15 client.5260.objecter _session_command_op_assign -1 2 2026-02-22T23:28:42.119 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 20 client.5260.objecter _get_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 20 client.5260.objecter put_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 20 client.5260.objecter _get_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 15 client.5260.objecter _session_command_op_remove -1 2 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 20 client.5260.objecter get_session s=0x7fd634080c40 osd=11 3 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 15 client.5260.objecter _session_command_op_assign 11 2 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 20 client.5260.objecter put_session s=0x7fd634080c40 osd=11 4 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 10 client.5260.objecter _send_command 2 2026-02-22T23:28:42.120 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.118+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 --> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd6640caa00 con 0x7fd634081310 2026-02-22T23:28:42.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd651ffb640 1 -- 10.20.193.6:0/2573351888 <== osd.11 v2:10.20.193.189:6802/2070530833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd6640caa00 con 0x7fd634081310 2026-02-22T23:28:42.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd651ffb640 10 client.5260.objecter ms_dispatch2 0x7fd664000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:42.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd651ffb640 10 client.5260.objecter _finish_command 2 = system:0 2026-02-22T23:28:42.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd651ffb640 20 client.5260.objecter put_session s=0x7fd634080c40 osd=11 4 2026-02-22T23:28:42.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd651ffb640 15 client.5260.objecter _session_command_op_remove 11 2 2026-02-22T23:28:42.127 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:42.127 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:42.127 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:42.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 10 librados: watch_flush enter 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 10 librados: watch_flush exit 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 10 client.5260.objecter close_session for osd.11 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fd634081310 msgr2=0x7fd634083730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6802/2070530833,v1:10.20.193.189:6803/2070530833] conn(0x7fd634081310 0x7fd634083730 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 20 client.5260.objecter put_session s=0x7fd634080c40 osd=11 1 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 20 client.5260.objecter shutdown clearing up homeless session... 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 10 client.5260.objecter successfully canceled tick 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd634078e30 msgr2=0x7fd63407b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fd634078e30 0x7fd63407b2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd654016ed0 tx=0x7fd654033000 comp rx=0 tx=0).stop 2026-02-22T23:28:42.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 10 monclient: shutdown 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd6641503e0 msgr2=0x7fd66415b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 --2- 10.20.193.6:0/2573351888 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fd6641503e0 0x7fd66415b060 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd65800c430 tx=0x7fd65800c900 comp rx=0 tx=0).stop 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd6698a2640 1 -- 10.20.193.6:0/2573351888 reap_dead start 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 shutdown_connections 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 >> 10.20.193.6:0/2573351888 conn(0x7fd6640cd600 msgr2=0x7fd6640cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 shutdown_connections 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 -- 10.20.193.6:0/2573351888 wait complete. 2026-02-22T23:28:42.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.125+0000 7fd66b32c640 1 librados: shutdown 2026-02-22T23:28:42.134 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2026-02-22T23:28:42.201 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.200+0000 7fe070258640 10 monclient: get_monmap_and_config 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.200+0000 7fe070258640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.200+0000 7fe070258640 10 monclient: monmap: 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:42.201066+0000 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:42.201066+0000 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:42.202 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:42.203 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:42.203 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:42.203 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:42.203 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.200+0000 7fe070258640 1 Processor -- start 2026-02-22T23:28:42.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.200+0000 7fe070258640 1 -- start start 2026-02-22T23:28:42.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.200+0000 7fe070258640 10 monclient: init 2026-02-22T23:28:42.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient: _add_conns ranks=[2,5,4,3,0,1] 2026-02-22T23:28:42.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fe06806ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:42.206 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fe0680c99a0 0x7fe068070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient(hunting): picked mon.noname-c con 0x7fe0680c99a0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe06815a5e0 0x7fe06814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient(hunting): picked mon.noname-b con 0x7fe06815a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0680c7e90 con 0x7fe06806ff20 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fe068159a40 con 0x7fe06815a5e0 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe0680c8170 con 0x7fe0680c99a0 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe070258640 10 monclient(hunting): authenticate will time out at 1396.928534s 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.201+0000 7fe06d7cc640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fe0680c99a0 0x7fe068070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.207 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe06815a5e0 0x7fe06814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fe06815a5e0 0x7fe06814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:43934/0 (socket says 10.20.193.6:43934) 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 1 -- 10.20.193.6:0/1117303887 learned_addr learned my addr 10.20.193.6:0/1117303887 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 10 monclient(hunting): get_auth_request con 0x7fe06815a5e0 auth_method 0 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06d7cc640 10 monclient(hunting): get_auth_request con 0x7fe0680c99a0 auth_method 0 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06d7cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06d7cc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06d7cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.208 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06e7ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06d7cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06d7cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.202+0000 7fe06d7cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 10 monclient(hunting): handle_auth_done global_id 5263 payload 274 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 1 -- 10.20.193.6:0/1117303887 >> v2:10.20.193.80:3300/0 conn(0x7fe0680c99a0 msgr2=0x7fe068070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 1 --2- 10.20.193.6:0/1117303887 >> v2:10.20.193.80:3300/0 conn(0x7fe0680c99a0 0x7fe068070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 1 -- 10.20.193.6:0/1117303887 >> v1:10.20.193.80:6789/0 conn(0x7fe06806ff20 legacy=0x7fe068159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 1 monclient: found mon.noname-b 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.209 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06e7ce640 1 -- 10.20.193.6:0/1117303887 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0680c8450 con 0x7fe06815a5e0 2026-02-22T23:28:42.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.203+0000 7fe06d7cc640 1 --2- 10.20.193.6:0/1117303887 >> v2:10.20.193.80:3300/0 conn(0x7fe0680c99a0 0x7fe068070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:42.210 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.204+0000 7fe06e7ce640 1 --2- 10.20.193.6:0/1117303887 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06815a5e0 0x7fe06814fe10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe064009e50 tx=0x7fe06402f2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5051017eab8c9a server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 1 -- 10.20.193.6:0/1117303887 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06402fd60 con 0x7fe06815a5e0 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: dump: 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.216 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 1 -- 10.20.193.6:0/1117303887 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0640373f0 con 0x7fe06815a5e0 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 5 monclient: authenticate success, global_id 5263 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:42.217 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 1 -- 10.20.193.6:0/1117303887 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640377c0 con 0x7fe06815a5e0 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe06cfcb640 10 monclient: dump: 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.218 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 4 set_mon_vals no callback set 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 10 monclient: shutdown 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 -- 10.20.193.6:0/1117303887 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06815a5e0 msgr2=0x7fe06814fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 --2- 10.20.193.6:0/1117303887 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06815a5e0 0x7fe06814fe10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe064009e50 tx=0x7fe06402f2a0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 -- 10.20.193.6:0/1117303887 shutdown_connections 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 --2- 10.20.193.6:0/1117303887 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06815a5e0 0x7fe06814fe10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 --2- 10.20.193.6:0/1117303887 >> v2:10.20.193.80:3300/0 conn(0x7fe0680c99a0 0x7fe068070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 -- 10.20.193.6:0/1117303887 >> 10.20.193.6:0/1117303887 conn(0x7fe0680cd780 msgr2=0x7fe0680cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.219 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 -- 10.20.193.6:0/1117303887 shutdown_connections 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.206+0000 7fe070258640 1 -- 10.20.193.6:0/1117303887 wait complete. 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 10 monclient: monmap: 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:42.220 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 1 librados: starting msgr at 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 1 librados: starting objecter 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 1 Processor -- start 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 1 -- start start 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 1 librados: setting wanted keys 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 1 librados: calling monclient init 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.207+0000 7fe070258640 10 monclient: init 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06806ff20 0x7fe0680c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): picked mon.noname-b con 0x7fe06806ff20 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0680c99a0 0x7fe0680c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.221 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): picked mon.noname-a con 0x7fe0680c99a0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe06815a5e0 0x7fe06814bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): picked mon.noname-c con 0x7fe06815a5e0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fe068150350 con 0x7fe0680c99a0 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fe0680c8a10 con 0x7fe06806ff20 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681504d0 con 0x7fe06815a5e0 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe070258640 10 monclient(hunting): authenticate will time out at 1396.935264s 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06dfcd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06806ff20 0x7fe0680c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06d7cc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0680c99a0 0x7fe0680c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.222 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06dfcd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06806ff20 0x7fe0680c00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.40:3300/0 says I am v2:10.20.193.6:43936/0 (socket says 10.20.193.6:43936) 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06e7ce640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe06815a5e0 0x7fe06814bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06dfcd640 1 -- 10.20.193.6:0/2560483761 learned_addr learned my addr 10.20.193.6:0/2560483761 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06dfcd640 10 monclient(hunting): get_auth_request con 0x7fe06806ff20 auth_method 0 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06dfcd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06dfcd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06dfcd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06d7cc640 10 monclient(hunting): get_auth_request con 0x7fe0680c99a0 auth_method 0 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06d7cc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06d7cc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.208+0000 7fe06d7cc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.223 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06e7ce640 10 monclient(hunting): get_auth_request con 0x7fe06815a5e0 auth_method 0 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06e7ce640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06e7ce640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06e7ce640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06dfcd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06dfcd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06dfcd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06d7cc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06d7cc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06d7cc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06e7ce640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06e7ce640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.224 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.209+0000 7fe06e7ce640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 10 monclient(hunting): handle_auth_done global_id 5266 payload 824 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 1 -- 10.20.193.6:0/2560483761 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe06815a5e0 msgr2=0x7fe06814bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe06815a5e0 0x7fe06814bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 1 -- 10.20.193.6:0/2560483761 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0680c99a0 msgr2=0x7fe0680c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0680c99a0 0x7fe0680c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 1 monclient: found mon.noname-b 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06dfcd640 1 -- 10.20.193.6:0/2560483761 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06806f260 con 0x7fe06806ff20 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.210+0000 7fe06e7ce640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe06815a5e0 0x7fe06814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.211+0000 7fe06dfcd640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06806ff20 0x7fe0680c00b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe05800c5b0 tx=0x7fe05800ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.225 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.211+0000 7fe0567fc640 10 client.?.objecter ms_handle_connect 0x7fe06806ff20 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.211+0000 7fe0567fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 1 -- 10.20.193.6:0/2560483761 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580052d0 con 0x7fe06806ff20 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: dump: 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.226 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 1 -- 10.20.193.6:0/2560483761 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe058005470 con 0x7fe06806ff20 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 1 -- 10.20.193.6:0/2560483761 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580038f0 con 0x7fe06806ff20 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe057fff640 4 set_mon_vals no callback set 2026-02-22T23:28:42.227 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe0567fc640 10 monclient: dump: 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 5 monclient: authenticate success, global_id 5266 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 monclient: _renew_subs 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.228 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0680bf010 con 0x7fe06806ff20 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 client.5266.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 monclient: _renew_subs 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0680bf5d0 con 0x7fe06806ff20 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 1 librados: init done 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.229 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.213+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe068150860 con 0x7fe06806ff20 2026-02-22T23:28:42.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.225+0000 7fe0567fc640 1 -- 10.20.193.6:0/2560483761 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fe058003a90 con 0x7fe06806ff20 2026-02-22T23:28:42.230 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.225+0000 7fe0567fc640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe038078e50 0x7fe03807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.226+0000 7fe06d7cc640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe038078e50 0x7fe03807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.226+0000 7fe06d7cc640 10 monclient: get_auth_request con 0x7fe038078e50 auth_method 0 2026-02-22T23:28:42.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.226+0000 7fe06d7cc640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe038078e50 0x7fe03807b310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009660 tx=0x7fe05c009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.231 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.226+0000 7fe0567fc640 10 client.5266.objecter ms_handle_connect 0x7fe038078e50 2026-02-22T23:28:42.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.320+0000 7fe0567fc640 1 -- 10.20.193.6:0/2560483761 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fe05805e020 con 0x7fe06806ff20 2026-02-22T23:28:42.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.320+0000 7fe0567fc640 10 client.5266.objecter ms_dispatch2 0x7fe068000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:42.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.320+0000 7fe0567fc640 3 client.5266.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:42.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.320+0000 7fe0567fc640 3 client.5266.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:42.321 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.320+0000 7fe0567fc640 20 client.5266.objecter dump_active .. 0 homeless 2026-02-22T23:28:42.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.325+0000 7fe0567fc640 1 -- 10.20.193.6:0/2560483761 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fe058061fe0 con 0x7fe06806ff20 2026-02-22T23:28:42.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.325+0000 7fe0567fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.327 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.325+0000 7fe0567fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:42.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.409+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "11", "target": ["mon-mgr", ""]}) -- 0x7fe0681583f0 con 0x7fe038078e50 2026-02-22T23:28:42.411 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe0567fc640 1 -- 10.20.193.6:0/2560483761 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 11 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7fe0681583f0 con 0x7fe038078e50 2026-02-22T23:28:42.411 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 11 to deep-scrub 2026-02-22T23:28:42.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 10 librados: watch_flush enter 2026-02-22T23:28:42.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 10 librados: watch_flush exit 2026-02-22T23:28:42.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 20 client.5266.objecter shutdown clearing up homeless session... 2026-02-22T23:28:42.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 10 client.5266.objecter successfully canceled tick 2026-02-22T23:28:42.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe038078e50 msgr2=0x7fe03807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe038078e50 0x7fe03807b310 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009660 tx=0x7fe05c009290 comp rx=0 tx=0).stop 2026-02-22T23:28:42.413 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 10 monclient: shutdown 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06806ff20 msgr2=0x7fe0680c00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06806ff20 0x7fe0680c00b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe05800c5b0 tx=0x7fe05800ca80 comp rx=0 tx=0).stop 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.410+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 shutdown_connections 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fe038078e50 0x7fe03807b310 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fe06815a5e0 0x7fe06814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fe0680c99a0 0x7fe0680c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 --2- 10.20.193.6:0/2560483761 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fe06806ff20 0x7fe0680c00b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 >> 10.20.193.6:0/2560483761 conn(0x7fe0680cd780 msgr2=0x7fe06814e500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 shutdown_connections 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 -- 10.20.193.6:0/2560483761 wait complete. 2026-02-22T23:28:42.414 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.411+0000 7fe070258640 1 librados: shutdown 2026-02-22T23:28:42.420 INFO:tasks.ceph:Scrubbing osd.15 2026-02-22T23:28:42.420 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:28:42.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.489+0000 7fcc7231b640 10 monclient: get_monmap_and_config 2026-02-22T23:28:42.490 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.489+0000 7fcc7231b640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.489+0000 7fcc7231b640 10 monclient: monmap: 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:42.490107+0000 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:42.490107+0000 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:42.491 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:42.492 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:42.492 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.489+0000 7fcc7231b640 1 Processor -- start 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.489+0000 7fcc7231b640 1 -- start start 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.489+0000 7fcc7231b640 10 monclient: init 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient: _add_conns ranks=[5,0,3,4,1,2] 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fcc6c1503e0 0x7fcc6c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient(hunting): picked mon.noname-c con 0x7fcc6c1503e0 addr v2:10.20.193.80:3300/0 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fcc6c15a070 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:42.495 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcc6c0704c0 0x7fcc6c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient(hunting): picked mon.noname-a con 0x7fcc6c0704c0 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc6c15a570 con 0x7fcc6c15a070 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7fcc6c0c7d70 con 0x7fcc6c0704c0 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 1 -- --> v2:10.20.193.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcc6c0c8610 con 0x7fcc6c1503e0 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc7231b640 10 monclient(hunting): authenticate will time out at 1397.217484s 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc6bfff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fcc6c1503e0 0x7fcc6c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.490+0000 7fcc70891640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7fcc6c0704c0 0x7fcc6c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 1 --2- >> v2:10.20.193.80:3300/0 conn(0x7fcc6c1503e0 0x7fcc6c159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:43338/0 (socket says 10.20.193.6:43338) 2026-02-22T23:28:42.496 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 1 -- 10.20.193.6:0/1668120913 learned_addr learned my addr 10.20.193.6:0/1668120913 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 10 monclient(hunting): get_auth_request con 0x7fcc6c1503e0 auth_method 0 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc70891640 10 monclient(hunting): get_auth_request con 0x7fcc6c0704c0 auth_method 0 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc70891640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc70891640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc70891640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc70891640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.497 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc70891640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.491+0000 7fcc70891640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 10 monclient(hunting): handle_auth_done global_id 5289 payload 274 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 1 -- 10.20.193.6:0/1668120913 >> v2:10.20.193.80:3300/0 conn(0x7fcc6c1503e0 msgr2=0x7fcc6c159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 1 --2- 10.20.193.6:0/1668120913 >> v2:10.20.193.80:3300/0 conn(0x7fcc6c1503e0 0x7fcc6c159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 1 -- 10.20.193.6:0/1668120913 >> v1:10.20.193.6:6789/0 conn(0x7fcc6c15a070 legacy=0x7fcc6c06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 1 monclient: found mon.noname-a 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.492+0000 7fcc70891640 1 -- 10.20.193.6:0/1668120913 --> v2:10.20.193.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c15a850 con 0x7fcc6c0704c0 2026-02-22T23:28:42.498 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.493+0000 7fcc70891640 1 --2- 10.20.193.6:0/1668120913 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c0c8aa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0ce490 tx=0x7fcc6002f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecd1112d70616ed server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.504 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 1 -- 10.20.193.6:0/1668120913 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6003c070 con 0x7fcc6c0704c0 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: dump: 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.505 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 1 -- 10.20.193.6:0/1668120913 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc60037440 con 0x7fcc6c0704c0 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 5 monclient: authenticate success, global_id 5289 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 1 -- 10.20.193.6:0/1668120913 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60037810 con 0x7fcc6c0704c0 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 4 set_mon_vals no callback set 2026-02-22T23:28:42.506 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc6affd640 10 monclient: dump: 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.507 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 10 monclient: shutdown 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 -- 10.20.193.6:0/1668120913 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 msgr2=0x7fcc6c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 --2- 10.20.193.6:0/1668120913 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c0c8aa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0ce490 tx=0x7fcc6002f250 comp rx=0 tx=0).stop 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 -- 10.20.193.6:0/1668120913 shutdown_connections 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 --2- 10.20.193.6:0/1668120913 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c0c8aa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 --2- 10.20.193.6:0/1668120913 >> v2:10.20.193.80:3300/0 conn(0x7fcc6c1503e0 0x7fcc6c159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 -- 10.20.193.6:0/1668120913 >> 10.20.193.6:0/1668120913 conn(0x7fcc6c0cd600 msgr2=0x7fcc6c0cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 -- 10.20.193.6:0/1668120913 shutdown_connections 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 -- 10.20.193.6:0/1668120913 wait complete. 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 10 monclient: monmap: 2026-02-22T23:28:42.508 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 librados: starting msgr at 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.496+0000 7fcc7231b640 1 librados: starting objecter 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 Processor -- start 2026-02-22T23:28:42.509 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 -- start start 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 librados: setting wanted keys 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 librados: calling monclient init 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient: init 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient: _add_conns ranks=[0,1,2] 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): picked mon.noname-a con 0x7fcc6c0704c0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc6c1503e0 0x7fcc6c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): picked mon.noname-b con 0x7fcc6c1503e0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc6c15a070 0x7fcc6c14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): picked mon.noname-c con 0x7fcc6c15a070 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.510 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1507c0 con 0x7fcc6c0704c0 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c0c82a0 con 0x7fcc6c1503e0 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c150940 con 0x7fcc6c15a070 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc7231b640 10 monclient(hunting): authenticate will time out at 1397.224463s 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6b7fe640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc6c1503e0 0x7fcc6c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6bfff640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37146/0 (socket says 10.20.193.6:37146) 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6bfff640 1 -- 10.20.193.6:0/1322805802 learned_addr learned my addr 10.20.193.6:0/1322805802 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6bfff640 10 monclient(hunting): get_auth_request con 0x7fcc6c0704c0 auth_method 0 2026-02-22T23:28:42.511 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc6bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.497+0000 7fcc70891640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc6c15a070 0x7fcc6c14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6b7fe640 10 monclient(hunting): get_auth_request con 0x7fcc6c1503e0 auth_method 0 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc70891640 10 monclient(hunting): get_auth_request con 0x7fcc6c15a070 auth_method 0 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc70891640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.512 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc70891640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc70891640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc6b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc70891640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc70891640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.498+0000 7fcc70891640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 10 monclient(hunting): handle_auth_done global_id 5292 payload 824 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 1 -- 10.20.193.6:0/1322805802 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc6c15a070 msgr2=0x7fcc6c14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fcc6c15a070 0x7fcc6c14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 1 -- 10.20.193.6:0/1322805802 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc6c1503e0 msgr2=0x7fcc6c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fcc6c1503e0 0x7fcc6c0ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.513 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 1 monclient: found mon.noname-a 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.499+0000 7fcc6bfff640 1 -- 10.20.193.6:0/1322805802 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c155e00 con 0x7fcc6c0704c0 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.500+0000 7fcc6bfff640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c072cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e930 tx=0x7fcc5800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.500+0000 7fcc68ff9640 10 client.?.objecter ms_handle_connect 0x7fcc6c0704c0 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.500+0000 7fcc68ff9640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.501+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58017070 con 0x7fcc6c0704c0 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.501+0000 7fcc68ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: dump: 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.514 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5800ca00 con 0x7fcc6c0704c0 2026-02-22T23:28:42.515 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 5 monclient: authenticate success, global_id 5292 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800cdf0 con 0x7fcc6c0704c0 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc6a7fc640 4 set_mon_vals no callback set 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc68ff9640 10 monclient: dump: 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.516 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 monclient: _renew_subs 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c1560e0 con 0x7fcc6c0704c0 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 client.5292.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 monclient: _renew_subs 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c1566a0 con 0x7fcc6c0704c0 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 1 librados: init done 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 client.5292.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.517 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 15 client.5292.objecter _session_command_op_assign -1 1 2026-02-22T23:28:42.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 20 client.5292.objecter _get_session osd=-1 returning homeless 2026-02-22T23:28:42.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 client.5292.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:42.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:42.518 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.502+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc6c159b30 con 0x7fcc6c0704c0 2026-02-22T23:28:42.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.514+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fcc58026480 con 0x7fcc6c0704c0 2026-02-22T23:28:42.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.514+0000 7fcc68ff9640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc3c078e00 0x7fcc3c07b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.515+0000 7fcc6b7fe640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc3c078e00 0x7fcc3c07b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.515+0000 7fcc6b7fe640 10 monclient: get_auth_request con 0x7fcc3c078e00 auth_method 0 2026-02-22T23:28:42.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.515+0000 7fcc6b7fe640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc3c078e00 0x7fcc3c07b2c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c14f060 tx=0x7fcc5c009110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.515+0000 7fcc68ff9640 10 client.5292.objecter ms_handle_connect 0x7fcc3c078e00 2026-02-22T23:28:42.520 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fcc58097d40 con 0x7fcc6c0704c0 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 10 client.5292.objecter ms_dispatch2 0x7fcc6c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 3 client.5292.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 3 client.5292.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 10 client.5292.objecter checking command 1 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7fcc3c0812e0 0x7fcc3c083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 20 client.5292.objecter _get_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 20 client.5292.objecter put_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 15 client.5292.objecter _session_command_op_remove -1 1 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 20 client.5292.objecter _get_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 20 client.5292.objecter get_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 15 client.5292.objecter _session_command_op_assign 15 1 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 20 client.5292.objecter put_session s=0x7fcc3c080c10 osd=15 4 2026-02-22T23:28:42.521 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 10 client.5292.objecter _send_command 1 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 --> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc3c083dd0 con 0x7fcc3c0812e0 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc68ff9640 20 client.5292.objecter dump_active .. 0 homeless 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.516+0000 7fcc70891640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7fcc3c0812e0 0x7fcc3c083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.517+0000 7fcc70891640 10 monclient: get_auth_request con 0x7fcc3c0812e0 auth_method 0 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.517+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcc5805e770 con 0x7fcc6c0704c0 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.517+0000 7fcc68ff9640 10 monclient: handle_get_version_reply finishing 1 version 28 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.518+0000 7fcc70891640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7fcc3c0812e0 0x7fcc3c083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.15 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.522 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.518+0000 7fcc68ff9640 10 client.5292.objecter ms_handle_connect 0x7fcc3c0812e0 2026-02-22T23:28:42.529 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.528+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== osd.15 v2:10.20.193.189:6826/3776650816 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27504 (crc 0 0 0) 0x7fcc3c083dd0 con 0x7fcc3c0812e0 2026-02-22T23:28:42.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.528+0000 7fcc68ff9640 10 client.5292.objecter ms_dispatch2 0x7fcc6c000b90 command_reply(tid 1: 0 ) 2026-02-22T23:28:42.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.528+0000 7fcc68ff9640 10 client.5292.objecter _finish_command 1 = system:0 2026-02-22T23:28:42.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.528+0000 7fcc68ff9640 20 client.5292.objecter put_session s=0x7fcc3c080c10 osd=15 4 2026-02-22T23:28:42.530 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.528+0000 7fcc68ff9640 15 client.5292.objecter _session_command_op_remove 15 1 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 10 client.5292.objecter _submit_command 2 [{"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}] 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 15 client.5292.objecter _session_command_op_assign -1 2 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 20 client.5292.objecter _get_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 20 client.5292.objecter put_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 20 client.5292.objecter _get_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 15 client.5292.objecter _session_command_op_remove -1 2 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 20 client.5292.objecter get_session s=0x7fcc3c080c10 osd=15 3 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 15 client.5292.objecter _session_command_op_assign 15 2 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 20 client.5292.objecter put_session s=0x7fcc3c080c10 osd=15 4 2026-02-22T23:28:42.536 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 10 client.5292.objecter _send_command 2 2026-02-22T23:28:42.537 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.534+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 --> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcc6c1574e0 con 0x7fcc3c0812e0 2026-02-22T23:28:42.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc68ff9640 1 -- 10.20.193.6:0/1322805802 <== osd.15 v2:10.20.193.189:6826/3776650816 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcc6c1574e0 con 0x7fcc3c0812e0 2026-02-22T23:28:42.542 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc68ff9640 10 client.5292.objecter ms_dispatch2 0x7fcc6c000b90 command_reply(tid 2: 0 ) 2026-02-22T23:28:42.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc68ff9640 10 client.5292.objecter _finish_command 2 = system:0 2026-02-22T23:28:42.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc68ff9640 20 client.5292.objecter put_session s=0x7fcc3c080c10 osd=15 4 2026-02-22T23:28:42.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc68ff9640 15 client.5292.objecter _session_command_op_remove 15 2 2026-02-22T23:28:42.543 INFO:teuthology.orchestra.run.trial006.stdout:{ 2026-02-22T23:28:42.543 INFO:teuthology.orchestra.run.trial006.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:28:42.543 INFO:teuthology.orchestra.run.trial006.stdout:} 2026-02-22T23:28:42.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 10 librados: watch_flush enter 2026-02-22T23:28:42.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 10 librados: watch_flush exit 2026-02-22T23:28:42.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 10 client.5292.objecter close_session for osd.15 2026-02-22T23:28:42.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7fcc3c0812e0 msgr2=0x7fcc3c083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6826/3776650816,v1:10.20.193.189:6827/3776650816] conn(0x7fcc3c0812e0 0x7fcc3c083700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 20 client.5292.objecter put_session s=0x7fcc3c080c10 osd=15 1 2026-02-22T23:28:42.545 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 20 client.5292.objecter shutdown clearing up homeless session... 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 10 client.5292.objecter successfully canceled tick 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc3c078e00 msgr2=0x7fcc3c07b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fcc3c078e00 0x7fcc3c07b2c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c14f060 tx=0x7fcc5c009110 comp rx=0 tx=0).stop 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 10 monclient: shutdown 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 msgr2=0x7fcc6c072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 --2- 10.20.193.6:0/1322805802 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fcc6c0704c0 0x7fcc6c072cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800e930 tx=0x7fcc5800ee00 comp rx=0 tx=0).stop 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc70891640 1 -- 10.20.193.6:0/1322805802 reap_dead start 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 shutdown_connections 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 >> 10.20.193.6:0/1322805802 conn(0x7fcc6c0cd600 msgr2=0x7fcc6c0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 shutdown_connections 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 -- 10.20.193.6:0/1322805802 wait complete. 2026-02-22T23:28:42.546 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.542+0000 7fcc7231b640 1 librados: shutdown 2026-02-22T23:28:42.551 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2026-02-22T23:28:42.619 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.618+0000 7fb891c74640 10 monclient: get_monmap_and_config 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.618+0000 7fb891c74640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.618+0000 7fb891c74640 10 monclient: monmap: 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:42.619066+0000 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:42.619066+0000 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:42.620 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:42.621 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:42.621 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:42.621 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.618+0000 7fb891c74640 1 Processor -- start 2026-02-22T23:28:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.618+0000 7fb891c74640 1 -- start start 2026-02-22T23:28:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.618+0000 7fb891c74640 10 monclient: init 2026-02-22T23:28:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 10 monclient: _add_conns ranks=[2,1,0,5,3,4] 2026-02-22T23:28:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7fb88c06ff20 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:42.623 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fb88c0c99a0 addr v1:10.20.193.40:6789/0 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fb88c15a5e0 addr v1:10.20.193.6:6789/0 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb88c0c7e90 con 0x7fb88c15a5e0 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb88c0c8170 con 0x7fb88c0c99a0 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb88c0c8450 con 0x7fb88c06ff20 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb891c74640 10 monclient(hunting): authenticate will time out at 1397.346426s 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb88affd640 1 --1- >> v1:10.20.193.40:6789/0 conn(0x7fb88c0c99a0 0x7fb88c14fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.40:6789/0 says I am v1:10.20.193.6:38398/0 (socket says 10.20.193.6:38398) 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb88affd640 1 -- 10.20.193.6:0/2666284109 learned_addr learned my addr 10.20.193.6:0/2666284109 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.619+0000 7fb88bfff640 1 --1- 10.20.193.6:0/2666284109 >> v1:10.20.193.6:6789/0 conn(0x7fb88c15a5e0 0x7fb88c0c6a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:52296/0 (socket says 10.20.193.6:52296) 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923308324 0 0) 0x7fb88c0c8450 con 0x7fb88c06ff20 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:28:42.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): my global_id is 5429 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb870003620 con 0x7fb88c06ff20 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737140694 0 0) 0x7fb88c0c7e90 con 0x7fb88c15a5e0 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): my global_id is 5295 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb88c0c8450 con 0x7fb88c15a5e0 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411339758 0 0) 0x7fb88c0c8170 con 0x7fb88c0c99a0 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _negotiate 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.625 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 10 monclient(hunting): my global_id is 5272 2026-02-22T23:28:42.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.621+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb88c0c7e90 con 0x7fb88c0c99a0 2026-02-22T23:28:42.633 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.2 v1:10.20.193.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3497287688 0 0) 0x7fb870003620 con 0x7fb88c06ff20 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 >> v1:10.20.193.40:6789/0 conn(0x7fb88c0c99a0 legacy=0x7fb88c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 >> v1:10.20.193.6:6789/0 conn(0x7fb88c15a5e0 legacy=0x7fb88c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 monclient: found mon.noname-c-legacy 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: _send_mon_message to mon.noname-c-legacy at v1:10.20.193.80:6789/0 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 --> v1:10.20.193.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c0c8730 con 0x7fb88c06ff20 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.634 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.2 v1:10.20.193.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fb878002f00 con 0x7fb88c06ff20 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb891c74640 5 monclient: authenticate success, global_id 5429 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: got monmap 1 from mon.noname-c-legacy (according to old e1) 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: dump: 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.635 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 monclient: mon.c has (v2) addrs [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] but i'm connected to v1:10.20.193.80:6789/0, reconnecting 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 >> v1:10.20.193.80:6789/0 conn(0x7fb88c06ff20 legacy=0x7fb88c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb870004030 0x7fb870004900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): picked mon.b con 0x7fb870004030 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb8700052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): picked mon.a con 0x7fb870004e40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 0x7fb87000bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.636 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): picked mon.c con 0x7fb870009ab0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb87000c570 con 0x7fb870004e40 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb87000c720 con 0x7fb870004030 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb87000c920 con 0x7fb870009ab0 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb88a7fc640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.622+0000 7fb891c74640 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.623+0000 7fb88affd640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb8700052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.623+0000 7fb88b7fe640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb870004030 0x7fb870004900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.623+0000 7fb88bfff640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 0x7fb87000bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.637 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.623+0000 7fb88affd640 10 monclient(hunting): get_auth_request con 0x7fb870004e40 auth_method 0 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.623+0000 7fb88affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.623+0000 7fb88affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.623+0000 7fb88affd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88b7fe640 10 monclient(hunting): get_auth_request con 0x7fb870004030 auth_method 0 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88b7fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88affd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.638 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88bfff640 1 -- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 msgr2=0x7fb87000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88bfff640 1 -- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 msgr2=0x7fb87000bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88bfff640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 0x7fb87000bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.624+0000 7fb88bfff640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 0x7fb87000bea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 10 monclient(hunting): handle_auth_done global_id 5429 payload 293 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 1 -- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 msgr2=0x7fb87000bea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb870009ab0 0x7fb87000bea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 1 -- 10.20.193.6:0/2666284109 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb870004030 msgr2=0x7fb870004900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb870004030 0x7fb870004900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 1 monclient: found mon.a 2026-02-22T23:28:42.639 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88bfff640 1 -- 10.20.193.6:0/2666284109 reap_dead start 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 1 -- 10.20.193.6:0/2666284109 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb870003620 con 0x7fb870004e40 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.625+0000 7fb88affd640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb8700052c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0031b0 tx=0x7fb87c02eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e757b134c2ad33 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c002dd0 con 0x7fb870004e40 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 10 monclient: dump: 2026-02-22T23:28:42.640 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb891c74640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87c033070 con 0x7fb870004e40 2026-02-22T23:28:42.641 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 1 -- 10.20.193.6:0/2666284109 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c004de0 con 0x7fb870004e40 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb88a7fc640 10 monclient: dump: 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.642 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb891c74640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.627+0000 7fb891c74640 4 set_mon_vals no callback set 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 10 monclient: shutdown 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 -- 10.20.193.6:0/2666284109 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 msgr2=0x7fb8700052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb8700052c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0031b0 tx=0x7fb87c02eed0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 -- 10.20.193.6:0/2666284109 shutdown_connections 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 --2- 10.20.193.6:0/2666284109 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb8700052c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 -- 10.20.193.6:0/2666284109 >> 10.20.193.6:0/2666284109 conn(0x7fb88c0cd780 msgr2=0x7fb88c0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 -- 10.20.193.6:0/2666284109 shutdown_connections 2026-02-22T23:28:42.643 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 -- 10.20.193.6:0/2666284109 wait complete. 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 10 monclient: monmap: 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.644 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 librados: starting msgr at 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 librados: starting objecter 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 Processor -- start 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 -- start start 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 librados: setting wanted keys 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 1 librados: calling monclient init 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.628+0000 7fb891c74640 10 monclient: init 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb88c073dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): picked mon.noname-a con 0x7fb870004e40 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb88c145cb0 0x7fb88c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.645 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): picked mon.noname-c con 0x7fb88c145cb0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb88c1f1550 0x7fb88c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): picked mon.noname-b con 0x7fb88c1f1550 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c070300 con 0x7fb870004e40 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c159a40 con 0x7fb88c1f1550 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c070480 con 0x7fb88c145cb0 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb891c74640 10 monclient(hunting): authenticate will time out at 1397.356353s 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb88c073dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.646 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88bfff640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb88c1f1550 0x7fb88c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88affd640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb88c145cb0 0x7fb88c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88b7fe640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb88c073dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37164/0 (socket says 10.20.193.6:37164) 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88b7fe640 1 -- 10.20.193.6:0/4150621066 learned_addr learned my addr 10.20.193.6:0/4150621066 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88b7fe640 10 monclient(hunting): get_auth_request con 0x7fb870004e40 auth_method 0 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88b7fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88b7fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88b7fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88bfff640 10 monclient(hunting): get_auth_request con 0x7fb88c1f1550 auth_method 0 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88bfff640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88bfff640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88bfff640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88affd640 10 monclient(hunting): get_auth_request con 0x7fb88c145cb0 auth_method 0 2026-02-22T23:28:42.647 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88affd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88affd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.629+0000 7fb88affd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88b7fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88b7fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88b7fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88bfff640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88affd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88affd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.630+0000 7fb88affd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 10 monclient(hunting): handle_auth_done global_id 5275 payload 824 2026-02-22T23:28:42.648 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 1 -- 10.20.193.6:0/4150621066 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb88c145cb0 msgr2=0x7fb88c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb88c145cb0 0x7fb88c06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 1 -- 10.20.193.6:0/4150621066 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 msgr2=0x7fb88c073dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb88c073dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 1 monclient: found mon.noname-b 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.631+0000 7fb88bfff640 1 -- 10.20.193.6:0/4150621066 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c06e540 con 0x7fb88c1f1550 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.632+0000 7fb88bfff640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb88c1f1550 0x7fb88c0cc430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb88c15a9c0 tx=0x7fb88000b0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.632+0000 7fb867fff640 10 client.?.objecter ms_handle_connect 0x7fb88c1f1550 2026-02-22T23:28:42.649 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.632+0000 7fb867fff640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:42.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 1 -- 10.20.193.6:0/4150621066 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880019070 con 0x7fb88c1f1550 2026-02-22T23:28:42.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:42.652 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: dump: 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.653 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 1 -- 10.20.193.6:0/4150621066 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88000bd10 con 0x7fb88c1f1550 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 5 monclient: authenticate success, global_id 5275 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 monclient: _renew_subs 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88c06d280 con 0x7fb88c1f1550 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 1 -- 10.20.193.6:0/4150621066 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880012610 con 0x7fb88c1f1550 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb867fff640 10 monclient: dump: 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.654 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 client.5275.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb889ffb640 4 set_mon_vals no callback set 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 monclient: _renew_subs 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88c1f43b0 con 0x7fb88c1f1550 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 1 librados: init done 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 10 monclient: _send_mon_message to mon.b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.655 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.634+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88c159ed0 con 0x7fb88c1f1550 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.646+0000 7fb867fff640 1 -- 10.20.193.6:0/4150621066 <== mon.1 v2:10.20.193.40:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fb8800127b0 con 0x7fb88c1f1550 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.646+0000 7fb867fff640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb85c078e50 0x7fb85c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.646+0000 7fb88b7fe640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb85c078e50 0x7fb85c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.646+0000 7fb88b7fe640 10 monclient: get_auth_request con 0x7fb85c078e50 auth_method 0 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.647+0000 7fb88b7fe640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb85c078e50 0x7fb85c07b310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb8780028f0 tx=0x7fb878009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.647+0000 7fb867fff640 10 client.5275.objecter ms_handle_connect 0x7fb85c078e50 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.647+0000 7fb867fff640 1 -- 10.20.193.6:0/4150621066 <== mon.1 v2:10.20.193.40:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fb88005dee0 con 0x7fb88c1f1550 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.647+0000 7fb867fff640 10 client.5275.objecter ms_dispatch2 0x7fb88c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.647+0000 7fb867fff640 3 client.5275.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.647+0000 7fb867fff640 3 client.5275.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:42.656 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.647+0000 7fb867fff640 20 client.5275.objecter dump_active .. 0 homeless 2026-02-22T23:28:42.749 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.749+0000 7fb867fff640 1 -- 10.20.193.6:0/4150621066 <== mon.1 v2:10.20.193.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fb880023080 con 0x7fb88c1f1550 2026-02-22T23:28:42.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.749+0000 7fb867fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:42.750 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.749+0000 7fb867fff640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:42.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.833+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "15", "target": ["mon-mgr", ""]}) -- 0x7fb88c072790 con 0x7fb85c078e50 2026-02-22T23:28:42.835 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.833+0000 7fb867fff640 1 -- 10.20.193.6:0/4150621066 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 15 to deep-scrub) ==== 42+0+0 (secure 0 0 0) 0x7fb88c072790 con 0x7fb85c078e50 2026-02-22T23:28:42.835 INFO:teuthology.orchestra.run.trial006.stderr:instructed osd(s) 15 to deep-scrub 2026-02-22T23:28:42.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 10 librados: watch_flush enter 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 10 librados: watch_flush exit 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 20 client.5275.objecter shutdown clearing up homeless session... 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 10 client.5275.objecter successfully canceled tick 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb85c078e50 msgr2=0x7fb85c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb85c078e50 0x7fb85c07b310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb8780028f0 tx=0x7fb878009040 comp rx=0 tx=0).stop 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 10 monclient: shutdown 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb88c1f1550 msgr2=0x7fb88c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.834+0000 7fb891c74640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb88c1f1550 0x7fb88c0cc430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb88c15a9c0 tx=0x7fb88000b0d0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 shutdown_connections 2026-02-22T23:28:42.837 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fb85c078e50 0x7fb85c07b310 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fb88c1f1550 0x7fb88c0cc430 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fb88c145cb0 0x7fb88c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 --2- 10.20.193.6:0/4150621066 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fb870004e40 0x7fb88c073dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 >> 10.20.193.6:0/4150621066 conn(0x7fb88c0cd780 msgr2=0x7fb88c0c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 shutdown_connections 2026-02-22T23:28:42.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 -- 10.20.193.6:0/4150621066 wait complete. 2026-02-22T23:28:42.838 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.835+0000 7fb891c74640 1 librados: shutdown 2026-02-22T23:28:42.845 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.910+0000 7f304b577640 10 monclient: get_monmap_and_config 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.910+0000 7f304b577640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.910+0000 7f304b577640 10 monclient: monmap: 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:28:42.911347+0000 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:28:42.911347+0000 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.912 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.913 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:28:42.913 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:28:42.913 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:28:42.913 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:28:42.913 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:28:42.913 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:28:42.913 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 1 Processor -- start 2026-02-22T23:28:42.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 1 -- start start 2026-02-22T23:28:42.916 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient: init 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient: _add_conns ranks=[4,3,2,1,5,0] 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f304c0c8fe0 0x7f304c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient(hunting): picked mon.noname-b con 0x7f304c0c8fe0 addr v2:10.20.193.40:3300/0 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f304c06d930 0x7f304c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient(hunting): picked mon.noname-a con 0x7f304c06d930 addr v2:10.20.193.6:3300/0 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f304c06dd80 addr v1:10.20.193.80:6789/0 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f304c159790 con 0x7f304c06dd80 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 1 -- --> v2:10.20.193.6:3300/0 -- mon_getmap magic: 0 -- 0x7f304c06e290 con 0x7f304c06d930 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.917 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f304c06e570 con 0x7f304c0c8fe0 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.911+0000 7f304b577640 10 monclient(hunting): authenticate will time out at 1397.638769s 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f304c0c8fe0 0x7f304c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 1 --2- >> v2:10.20.193.6:3300/0 conn(0x7f304c06d930 0x7f304c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304ad76640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f304c06dd80 0x7f304c159df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:46794/0 (socket says 10.20.193.6:46794) 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304ad76640 1 -- 10.20.193.6:0/353152641 learned_addr learned my addr 10.20.193.6:0/353152641 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 10 monclient(hunting): get_auth_request con 0x7f304c0c8fe0 auth_method 0 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 10 monclient(hunting): get_auth_request con 0x7f304c06d930 auth_method 0 2026-02-22T23:28:42.918 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.912+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f3049573640 1 -- 10.20.193.6:0/353152641 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322355081 0 0) 0x7f304c159790 con 0x7f304c06dd80 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f3049573640 10 monclient(hunting): _negotiate 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f3049573640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f3049573640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f3049573640 10 monclient(hunting): my global_id is 5435 2026-02-22T23:28:42.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f3049573640 1 -- 10.20.193.6:0/353152641 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3038003620 con 0x7f304c06dd80 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 10 monclient(hunting): handle_auth_done global_id 5278 payload 274 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 1 -- 10.20.193.6:0/353152641 >> v2:10.20.193.6:3300/0 conn(0x7f304c06d930 msgr2=0x7f304c15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 1 --2- 10.20.193.6:0/353152641 >> v2:10.20.193.6:3300/0 conn(0x7f304c06d930 0x7f304c15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 1 -- 10.20.193.6:0/353152641 >> v1:10.20.193.80:6789/0 conn(0x7f304c06dd80 legacy=0x7f304c159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 1 monclient: found mon.noname-b 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.913+0000 7f304a575640 1 -- 10.20.193.6:0/353152641 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c06e850 con 0x7f304c0c8fe0 2026-02-22T23:28:42.920 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.914+0000 7f304a575640 1 --2- 10.20.193.6:0/353152641 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c0c93c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30400098a0 tx=0x7f304002eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c49862094e3a4bbf server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:42.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 1 -- 10.20.193.6:0/353152641 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304003c070 con 0x7f304c0c8fe0 2026-02-22T23:28:42.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.925 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: dump: 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.926 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: _finish_auth 0 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: _check_auth_tickets 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 1 -- 10.20.193.6:0/353152641 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304002fb80 con 0x7f304c0c8fe0 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 5 monclient: authenticate success, global_id 5278 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 1 -- 10.20.193.6:0/353152641 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040037440 con 0x7f304c0c8fe0 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f3049573640 10 monclient: dump: 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:42.927 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 10 monclient: get_monmap_and_config success 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 4 set_mon_vals no callback set 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 10 monclient: shutdown 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:42.928 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 1 -- 10.20.193.6:0/353152641 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 msgr2=0x7f304c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.916+0000 7f304b577640 1 --2- 10.20.193.6:0/353152641 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c0c93c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30400098a0 tx=0x7f304002eeb0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 -- 10.20.193.6:0/353152641 shutdown_connections 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 --2- 10.20.193.6:0/353152641 >> v2:10.20.193.6:3300/0 conn(0x7f304c06d930 0x7f304c15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 --2- 10.20.193.6:0/353152641 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c0c93c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 -- 10.20.193.6:0/353152641 >> 10.20.193.6:0/353152641 conn(0x7f304c0cd480 msgr2=0x7f304c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 -- 10.20.193.6:0/353152641 shutdown_connections 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 -- 10.20.193.6:0/353152641 wait complete. 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 10 monclient: build_initial_monmap 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 10 monclient: monmap: 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:28:42.929 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 librados: starting msgr at 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 librados: starting objecter 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 Processor -- start 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 -- start start 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 librados: setting wanted keys 2026-02-22T23:28:42.930 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 1 librados: calling monclient init 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.917+0000 7f304b577640 10 monclient: init 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f304c1f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): picked mon.noname-a con 0x7f304c06d930 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 0x7f304c0c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): picked mon.noname-c con 0x7f304c06dd80 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): picked mon.noname-b con 0x7f304c0c8fe0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f304c06f0f0 con 0x7f304c06d930 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.931 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f304c06ee10 con 0x7f304c0c8fe0 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f304c15a500 con 0x7f304c06dd80 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304b577640 10 monclient(hunting): authenticate will time out at 1397.645155s 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f304c1f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f3049d74640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 0x7f304c0c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f304c1f41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:37184/0 (socket says 10.20.193.6:37184) 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 1 -- 10.20.193.6:0/3383620895 learned_addr learned my addr 10.20.193.6:0/3383620895 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 10 monclient(hunting): get_auth_request con 0x7f304c06d930 auth_method 0 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.932 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 10 monclient(hunting): get_auth_request con 0x7f304c0c8fe0 auth_method 0 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f3049d74640 10 monclient(hunting): get_auth_request con 0x7f304c06dd80 auth_method 0 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f3049d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f3049d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f3049d74640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.933 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.918+0000 7f304ad76640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.919+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.919+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.919+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 10 monclient(hunting): handle_auth_done global_id 5281 payload 824 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 msgr2=0x7f304c0c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 0x7f304c0c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 msgr2=0x7f304c1f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f304c1f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 10 monclient: _finish_hunting 0 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 1 monclient: found mon.noname-b 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:42.934 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:28:42.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304ad76640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304c1535e0 con 0x7f304c0c8fe0 2026-02-22T23:28:42.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.920+0000 7f304a575640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f304c1f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:28:42.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.921+0000 7f304ad76640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 msgr2=0x7f304c157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2026-02-22T23:28:42.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.921+0000 7f304ad76640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 msgr2=0x7f304c157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2026-02-22T23:28:42.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.921+0000 7f304ad76640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f303c00ca80 tx=0x7f303c00cf50 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2026-02-22T23:28:42.935 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:42.921+0000 7f304ad76640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2026-02-22T23:28:43.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304ad76640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:43.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304ad76640 10 monclient: get_auth_request con 0x7f304c0c8fe0 auth_method 0 2026-02-22T23:28:43.126 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304ad76640 0 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304ad76640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient: ms_handle_reset current mon [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient: _reopen_session rank -1 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 msgr2=0x7f304c157680 unknown :-1 s=STATE_CLOSED l=1).mark_down 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f304c0c8fe0 0x7f304c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient: _add_conns ranks=[0,2,1] 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f30180055b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient(hunting): picked mon.noname-a con 0x7f304c06d930 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 0x7f3018008060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient(hunting): picked mon.noname-c con 0x7f304c06dd80 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3018008ac0 0x7f301800af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:43.127 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient(hunting): picked mon.noname-b con 0x7f3018008ac0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f301800ba50 con 0x7f304c06d930 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f301800bbd0 con 0x7f3018008ac0 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient(hunting): start opening mon connection 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f301800bdd0 con 0x7f304c06dd80 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3032ffd640 10 monclient(hunting): _renew_subs 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304a575640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f30180055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f3049d74640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 0x7f3018008060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304a575640 10 monclient(hunting): get_auth_request con 0x7f304c06d930 auth_method 0 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304a575640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:43.128 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304a575640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.122+0000 7f304a575640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3018008ac0 0x7f301800af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f3049d74640 10 monclient(hunting): get_auth_request con 0x7f304c06dd80 auth_method 0 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f3049d74640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f3049d74640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f3049d74640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 10 monclient(hunting): get_auth_request con 0x7f3018008ac0 auth_method 0 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:43.129 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304a575640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f3049d74640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.123+0000 7f304ad76640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 10 monclient(hunting): handle_auth_done global_id 5281 payload 293 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 msgr2=0x7f3018008060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 0x7f3018008060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3018008ac0 msgr2=0x7f301800af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3018008ac0 0x7f301800af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.130 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 10 monclient: _finish_hunting 0 2026-02-22T23:28:43.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 1 monclient: found mon.noname-a 2026-02-22T23:28:43.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:28:43.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:28:43.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.124+0000 7f304a575640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301800c090 con 0x7f304c06d930 2026-02-22T23:28:43.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.125+0000 7f304a575640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f30180055b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f304003a040 tx=0x7f303c003220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:43.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.125+0000 7f3032ffd640 10 client.?.objecter ms_handle_connect 0x7f304c06d930 2026-02-22T23:28:43.131 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.125+0000 7f3032ffd640 10 client.?.objecter resend_mon_ops 2026-02-22T23:28:43.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304003c040 con 0x7f304c06d930 2026-02-22T23:28:43.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:43.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:28:43.133 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: dump: 2026-02-22T23:28:43.133 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:43.133 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:43.133 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: _finish_auth 0 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3040037a10 con 0x7f304c06d930 2026-02-22T23:28:43.134 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: handle_config config(0 keys) 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040037dc0 con 0x7f304c06d930 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 5 monclient: authenticate success, global_id 5281 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3048d72640 4 set_mon_vals no callback set 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 monclient: _renew_subs 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f304c1538c0 con 0x7f304c06d930 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f3032ffd640 10 monclient: dump: 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:43.135 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 client.5281.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 monclient: _renew_subs 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f304c153e80 con 0x7f304c06d930 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 1 librados: init done 2026-02-22T23:28:43.136 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:43.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:43.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:28:43.137 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.127+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304c0c93c0 con 0x7f304c06d930 2026-02-22T23:28:43.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.139+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f3040053070 con 0x7f304c06d930 2026-02-22T23:28:43.140 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.139+0000 7f3032ffd640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f304c0c8fe0 0x7f30180833c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:28:43.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.140+0000 7f3049d74640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f304c0c8fe0 0x7f30180833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:28:43.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.140+0000 7f3049d74640 10 monclient: get_auth_request con 0x7f304c0c8fe0 auth_method 0 2026-02-22T23:28:43.141 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.140+0000 7f3049d74640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f304c0c8fe0 0x7f30180833c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f304c1515d0 tx=0x7f3034009290 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:28:43.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.140+0000 7f3032ffd640 10 client.5281.objecter ms_handle_connect 0x7f304c0c8fe0 2026-02-22T23:28:43.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.141+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f3040082e60 con 0x7f304c06d930 2026-02-22T23:28:43.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.141+0000 7f3032ffd640 10 client.5281.objecter ms_dispatch2 0x7f304c000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:28:43.142 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.141+0000 7f3032ffd640 3 client.5281.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:28:43.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.141+0000 7f3032ffd640 3 client.5281.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:28:43.143 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.141+0000 7f3032ffd640 20 client.5281.objecter dump_active .. 0 homeless 2026-02-22T23:28:43.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.242+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f3040050020 con 0x7f304c06d930 2026-02-22T23:28:43.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.242+0000 7f3032ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:28:43.243 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.242+0000 7f3032ffd640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:28:43.324 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.323+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f304c14de30 con 0x7f304c0c8fe0 2026-02-22T23:28:43.326 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.324+0000 7f3032ffd640 1 -- 10.20.193.6:0/3383620895 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45246 (secure 0 0 0) 0x7f304c14de30 con 0x7f304c0c8fe0 2026-02-22T23:28:43.326 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 10 librados: watch_flush enter 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 10 librados: watch_flush exit 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 20 client.5281.objecter shutdown clearing up homeless session... 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 10 client.5281.objecter successfully canceled tick 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f304c0c8fe0 msgr2=0x7f30180833c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f304c0c8fe0 0x7f30180833c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f304c1515d0 tx=0x7f3034009290 comp rx=0 tx=0).stop 2026-02-22T23:28:43.329 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 10 monclient: shutdown 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 msgr2=0x7f30180055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f30180055b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f304003a040 tx=0x7f303c003220 comp rx=0 tx=0).stop 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 shutdown_connections 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f304c0c8fe0 0x7f30180833c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f3018008ac0 0x7f301800af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f304c06dd80 0x7f3018008060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 --2- 10.20.193.6:0/3383620895 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f304c06d930 0x7f30180055b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 >> 10.20.193.6:0/3383620895 conn(0x7f304c0cd480 msgr2=0x7f304c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 shutdown_connections 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 -- 10.20.193.6:0/3383620895 wait complete. 2026-02-22T23:28:43.330 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:28:43.325+0000 7f304b577640 1 librados: shutdown 2026-02-22T23:28:43.335 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2026-02-22T23:28:41.809086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":437144,"kb_used_data":5592,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512296,"statfs":{"total":1546188226560,"available":1545740591104,"internally_reserved":0,"allocated":5726208,"data_stored":2779510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25502,"internal_metadata":441424994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002177"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:36.952185+0000","last_change":"2026-02-22T23:23:26.541669+0000","last_active":"2026-02-22T23:25:36.952185+0000","last_peered":"2026-02-22T23:25:36.952185+0000","last_clean":"2026-02-22T23:25:36.952185+0000","last_became_active":"2026-02-22T23:23:24.999148+0000","last_became_peered":"2026-02-22T23:23:24.999148+0000","last_unstale":"2026-02-22T23:25:36.952185+0000","last_undegraded":"2026-02-22T23:25:36.952185+0000","last_fullsized":"2026-02-22T23:25:36.952185+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:33:54.198723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033743088999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:34.970590+0000","last_change":"2026-02-22T23:23:26.948831+0000","last_active":"2026-02-22T23:25:34.970590+0000","last_peered":"2026-02-22T23:25:34.970590+0000","last_clean":"2026-02-22T23:25:34.970590+0000","last_became_active":"2026-02-22T23:23:24.966693+0000","last_became_peered":"2026-02-22T23:23:24.966693+0000","last_unstale":"2026-02-22T23:25:34.970590+0000","last_undegraded":"2026-02-22T23:25:34.970590+0000","last_fullsized":"2026-02-22T23:25:34.970590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:30:16.311433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033152011000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":28,"state":"active+clean+scrubbing+deep","last_fresh":"2026-02-22T23:28:40.071179+0000","last_change":"2026-02-22T23:28:40.044975+0000","last_active":"2026-02-22T23:28:40.071179+0000","last_peered":"2026-02-22T23:28:40.071179+0000","last_clean":"2026-02-22T23:28:40.071179+0000","last_became_active":"2026-02-22T23:23:24.806260+0000","last_became_peered":"2026-02-22T23:23:24.806260+0000","last_unstale":"2026-02-22T23:28:40.071179+0000","last_undegraded":"2026-02-22T23:28:40.071179+0000","last_fullsized":"2026-02-22T23:28:40.071179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032290068999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:30.011204+0000","last_change":"2026-02-22T23:23:26.547663+0000","last_active":"2026-02-22T23:25:30.011204+0000","last_peered":"2026-02-22T23:25:30.011204+0000","last_clean":"2026-02-22T23:25:30.011204+0000","last_became_active":"2026-02-22T23:23:24.706162+0000","last_became_peered":"2026-02-22T23:23:24.706162+0000","last_unstale":"2026-02-22T23:25:30.011204+0000","last_undegraded":"2026-02-22T23:25:30.011204+0000","last_fullsized":"2026-02-22T23:25:30.011204+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:37:21.100237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041859805999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"27'2","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.979297+0000","last_change":"2026-02-22T23:23:26.386245+0000","last_active":"2026-02-22T23:25:31.979297+0000","last_peered":"2026-02-22T23:25:31.979297+0000","last_clean":"2026-02-22T23:25:31.979297+0000","last_became_active":"2026-02-22T23:23:25.054022+0000","last_became_peered":"2026-02-22T23:23:25.054022+0000","last_unstale":"2026-02-22T23:25:31.979297+0000","last_undegraded":"2026-02-22T23:25:31.979297+0000","last_fullsized":"2026-02-22T23:25:31.979297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:02:19.912782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.070943361999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:31.952620+0000","last_change":"2026-02-22T23:23:26.542103+0000","last_active":"2026-02-22T23:25:31.952620+0000","last_peered":"2026-02-22T23:25:31.952620+0000","last_clean":"2026-02-22T23:25:31.952620+0000","last_became_active":"2026-02-22T23:23:24.462347+0000","last_became_peered":"2026-02-22T23:23:24.462347+0000","last_unstale":"2026-02-22T23:25:31.952620+0000","last_undegraded":"2026-02-22T23:25:31.952620+0000","last_fullsized":"2026-02-22T23:25:31.952620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:09:34.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.033702211000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:26.977309+0000","last_change":"2026-02-22T23:23:26.382023+0000","last_active":"2026-02-22T23:25:26.977309+0000","last_peered":"2026-02-22T23:25:26.977309+0000","last_clean":"2026-02-22T23:25:26.977309+0000","last_became_active":"2026-02-22T23:23:24.464964+0000","last_became_peered":"2026-02-22T23:23:24.464964+0000","last_unstale":"2026-02-22T23:25:26.977309+0000","last_undegraded":"2026-02-22T23:25:26.977309+0000","last_fullsized":"2026-02-22T23:25:26.977309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:28:59.922786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055238717,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'1","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:27:37.033894+0000","last_change":"2026-02-22T23:23:26.538271+0000","last_active":"2026-02-22T23:27:37.033894+0000","last_peered":"2026-02-22T23:27:37.033894+0000","last_clean":"2026-02-22T23:27:37.033894+0000","last_became_active":"2026-02-22T23:23:24.503509+0000","last_became_peered":"2026-02-22T23:23:24.503509+0000","last_unstale":"2026-02-22T23:27:37.033894+0000","last_undegraded":"2026-02-22T23:27:37.033894+0000","last_fullsized":"2026-02-22T23:27:37.033894+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:22.984170+0000","last_clean_scrub_stamp":"2026-02-22T23:23:22.984170+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.032869134000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"27'32","reported_seq":66,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:25:21.948881+0000","last_change":"2026-02-22T23:23:22.760169+0000","last_active":"2026-02-22T23:25:21.948881+0000","last_peered":"2026-02-22T23:25:21.948881+0000","last_clean":"2026-02-22T23:25:21.948881+0000","last_became_active":"2026-02-22T23:23:22.740754+0000","last_became_peered":"2026-02-22T23:23:22.740754+0000","last_unstale":"2026-02-22T23:25:21.948881+0000","last_undegraded":"2026-02-22T23:25:21.948881+0000","last_fullsized":"2026-02-22T23:25:21.948881+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:23:20.759065+0000","last_clean_scrub_stamp":"2026-02-22T23:23:20.759065+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:14:46.099842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":23,"seq":98784247875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,9,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":15,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2,4,5,6,7,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":22,"seq":94489280580,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,3,4,5,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":6,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,4,5,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":21,"seq":90194313283,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[2,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":21,"seq":90194313282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313282,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":21,"seq":90194313283,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[0,4,6,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":21,"seq":90194313283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280579,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,5,6,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[2,4,5,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,5,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":22,"seq":94489280578,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":22,"seq":94489280579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:28:43.341 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.341 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.341 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.341 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.342 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.342 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.342 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.342 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-22T23:23:22.984170+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=22, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.342 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-22T23:23:20.759065+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=23, tm_sec=20, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=28, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:28:43.342 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-22T23:28:53.432 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:37.682 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.434 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:37.684 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.445 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:37.695 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.460 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:37.672 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.461 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:37.672 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.462 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:37.708 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.462 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:37.709 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.471 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:37.682 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:28:53.471 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:37.717 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:03.343 DEBUG:teuthology.orchestra.run.trial006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:29:03.415 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 10 monclient: get_monmap_and_config 2026-02-22T23:29:03.415 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 10 monclient: build_initial_monmap 2026-02-22T23:29:03.415 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 10 monclient: monmap: 2026-02-22T23:29:03.415 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:29:03.414661+0000 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:29:03.414661+0000 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:29:03.416 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:29:03.417 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 1 Processor -- start 2026-02-22T23:29:03.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 1 -- start start 2026-02-22T23:29:03.422 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 10 monclient: init 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 10 monclient: _reopen_session rank -1 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 10 monclient: _add_conns ranks=[1,0,4,5,2,3] 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.413+0000 7fec79790640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7fec7406ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 10 monclient(hunting): picked mon.noname-a-legacy con 0x7fec740c99a0 addr v1:10.20.193.6:6789/0 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7fec7415a5e0 0x7fec74070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 10 monclient(hunting): picked mon.noname-b con 0x7fec7415a5e0 addr v2:10.20.193.40:3300/0 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 1 -- --> v1:10.20.193.6:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec740c7e90 con 0x7fec740c99a0 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec740c8170 con 0x7fec7406ff20 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7fec74159a40 con 0x7fec7415a5e0 2026-02-22T23:29:03.423 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 10 monclient(hunting): _renew_subs 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec79790640 10 monclient(hunting): authenticate will time out at 1418.142038s 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec727fc640 1 --1- >> v1:10.20.193.6:6789/0 conn(0x7fec740c99a0 0x7fec7414fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.6:6789/0 says I am v1:10.20.193.6:38652/0 (socket says 10.20.193.6:38652) 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec727fc640 1 -- 10.20.193.6:0/3175223700 learned_addr learned my addr 10.20.193.6:0/3175223700 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.414+0000 7fec737fe640 1 --2- 10.20.193.6:0/3175223700 >> v2:10.20.193.40:3300/0 conn(0x7fec7415a5e0 0x7fec74070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 <== mon.0 v1:10.20.193.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1065150118 0 0) 0x7fec740c7e90 con 0x7fec740c99a0 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec71ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec71ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec71ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec71ffb640 10 monclient(hunting): my global_id is 5307 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 --> v1:10.20.193.6:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec60003620 con 0x7fec740c99a0 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec737fe640 10 monclient(hunting): get_auth_request con 0x7fec7415a5e0 auth_method 0 2026-02-22T23:29:03.424 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec737fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.415+0000 7fec737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 <== mon.1 v1:10.20.193.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439021979 0 0) 0x7fec740c8170 con 0x7fec7406ff20 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec71ffb640 10 monclient(hunting): _negotiate 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec71ffb640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec71ffb640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec71ffb640 10 monclient(hunting): my global_id is 5284 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.416+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 --> v1:10.20.193.40:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec740c7e90 con 0x7fec7406ff20 2026-02-22T23:29:03.425 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 <== mon.0 v1:10.20.193.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 338168446 0 0) 0x7fec60003620 con 0x7fec740c99a0 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 >> v2:10.20.193.40:3300/0 conn(0x7fec7415a5e0 msgr2=0x7fec74070300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 --2- 10.20.193.6:0/3175223700 >> v2:10.20.193.40:3300/0 conn(0x7fec7415a5e0 0x7fec74070300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 >> v1:10.20.193.40:6789/0 conn(0x7fec7406ff20 legacy=0x7fec74159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: _finish_hunting 0 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 monclient: found mon.noname-a-legacy 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: _send_mon_message to mon.noname-a-legacy at v1:10.20.193.6:6789/0 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 --> v1:10.20.193.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec740c8450 con 0x7fec740c99a0 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: _finish_auth 0 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: _check_auth_tickets 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:29:03.426 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 <== mon.0 v1:10.20.193.6:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2295204738 0 0) 0x7fec5c002f00 con 0x7fec740c99a0 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: got monmap 1 from mon.noname-a-legacy (according to old e1) 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: dump: 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.427 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 monclient: mon.a has (v2) addrs [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] but i'm connected to v1:10.20.193.6:6789/0, reconnecting 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: _reopen_session rank -1 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 >> v1:10.20.193.6:6789/0 conn(0x7fec740c99a0 legacy=0x7fec7414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec60003dd0 0x7fec600041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): picked mon.c con 0x7fec60003dd0 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60004900 0x7fec60004d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): picked mon.b con 0x7fec60004900 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec60005490 0x7fec6000bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): picked mon.a con 0x7fec60005490 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.428 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fec74159a40 con 0x7fec60005490 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000c380 con 0x7fec60004900 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000c580 con 0x7fec60003dd0 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): _renew_subs 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.417+0000 7fec71ffb640 10 monclient(hunting): _finish_auth 0 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec60003dd0 0x7fec600041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 10 monclient(hunting): get_auth_request con 0x7fec60003dd0 auth_method 0 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:29:03.429 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.418+0000 7fec72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec60005490 0x7fec6000bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 10 monclient(hunting): get_auth_request con 0x7fec60005490 auth_method 0 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 10 monclient(hunting): _init_auth already have auth, reseting 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 10 monclient(hunting): handle_auth_done global_id 5307 payload 293 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 1 -- 10.20.193.6:0/3175223700 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60004900 msgr2=0x7fec60004d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:29:03.430 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60004900 0x7fec60004d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 1 -- 10.20.193.6:0/3175223700 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec60005490 msgr2=0x7fec6000bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec60005490 0x7fec6000bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 10 monclient: _finish_hunting 0 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 1 monclient: found mon.c 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 1 -- 10.20.193.6:0/3175223700 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec60003620 con 0x7fec60003dd0 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 10 monclient: _finish_auth 0 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec737fe640 1 -- 10.20.193.6:0/3175223700 reap_dead start 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 10 monclient: _check_auth_tickets 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec79790640 5 monclient: authenticate success, global_id 5307 2026-02-22T23:29:03.431 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.419+0000 7fec79790640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.420+0000 7fec72ffd640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec60003dd0 0x7fec600041d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec68003210 tx=0x7fec6802ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=965c44aeadd3cd5d server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68002e30 con 0x7fec60003dd0 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 10 monclient: dump: 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.439 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec680052f0 con 0x7fec60003dd0 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 10 monclient: handle_config config(0 keys) 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 10 monclient: get_monmap_and_config success 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 1 -- 10.20.193.6:0/3175223700 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680068d0 con 0x7fec60003dd0 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.440 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 4 set_mon_vals no callback set 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec71ffb640 10 monclient: dump: 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.441 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 10 monclient: shutdown 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 1 -- 10.20.193.6:0/3175223700 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec60003dd0 msgr2=0x7fec600041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec60003dd0 0x7fec600041d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec68003210 tx=0x7fec6802ef10 comp rx=0 tx=0).stop 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 1 -- 10.20.193.6:0/3175223700 shutdown_connections 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 1 --2- 10.20.193.6:0/3175223700 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec60003dd0 0x7fec600041d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 1 -- 10.20.193.6:0/3175223700 >> 10.20.193.6:0/3175223700 conn(0x7fec740cd780 msgr2=0x7fec740cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 1 -- 10.20.193.6:0/3175223700 shutdown_connections 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.422+0000 7fec79790640 1 -- 10.20.193.6:0/3175223700 wait complete. 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 10 monclient: build_initial_monmap 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 10 monclient: monmap: 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:29:03.442 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 1 librados: starting msgr at 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 1 librados: starting objecter 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 1 Processor -- start 2026-02-22T23:29:03.443 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 1 -- start start 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 1 librados: setting wanted keys 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 1 librados: calling monclient init 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.423+0000 7fec79790640 10 monclient: init 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient: _reopen_session rank -1 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient: _add_conns ranks=[1,0,2] 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60003dd0 0x7fec741f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): picked mon.noname-b con 0x7fec60003dd0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec741488f0 0x7fec741f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): picked mon.noname-a con 0x7fec741488f0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec74148d70 0x7fec741f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): picked mon.noname-c con 0x7fec74148d70 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.444 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7fec74070300 con 0x7fec741488f0 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7fec740c8a10 con 0x7fec60003dd0 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7fec74070480 con 0x7fec74148d70 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): _renew_subs 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec79790640 10 monclient(hunting): authenticate will time out at 1418.152327s 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec727fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec741488f0 0x7fec741f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec737fe640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec74148d70 0x7fec741f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec72ffd640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60003dd0 0x7fec741f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec727fc640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec741488f0 0x7fec741f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.6:3300/0 says I am v2:10.20.193.6:56000/0 (socket says 10.20.193.6:56000) 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec727fc640 1 -- 10.20.193.6:0/1171937007 learned_addr learned my addr 10.20.193.6:0/1171937007 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:29:03.445 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec727fc640 10 monclient(hunting): get_auth_request con 0x7fec741488f0 auth_method 0 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec727fc640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec727fc640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec727fc640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec737fe640 10 monclient(hunting): get_auth_request con 0x7fec74148d70 auth_method 0 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec737fe640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec737fe640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec737fe640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec72ffd640 10 monclient(hunting): get_auth_request con 0x7fec60003dd0 auth_method 0 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec72ffd640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec72ffd640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.424+0000 7fec72ffd640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.446 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec727fc640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec727fc640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec727fc640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec737fe640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec72ffd640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec72ffd640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.425+0000 7fec72ffd640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 10 monclient(hunting): handle_auth_done global_id 5310 payload 824 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 1 -- 10.20.193.6:0/1171937007 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec74148d70 msgr2=0x7fec741f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec74148d70 0x7fec741f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 1 -- 10.20.193.6:0/1171937007 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60003dd0 msgr2=0x7fec741f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.447 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60003dd0 0x7fec741f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 10 monclient: _finish_hunting 0 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 1 monclient: found mon.noname-a 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 10 monclient: _send_mon_message to mon.noname-a at v2:10.20.193.6:3300/0 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec727fc640 1 -- 10.20.193.6:0/1171937007 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec741f4880 con 0x7fec741488f0 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.426+0000 7fec737fe640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec74148d70 0x7fec741f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec727fc640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec741488f0 0x7fec741f51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c730 tx=0x7fec5c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 client.?.objecter ms_handle_connect 0x7fec741488f0 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 client.?.objecter resend_mon_ops 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 1 -- 10.20.193.6:0/1171937007 <== mon.0 v2:10.20.193.6:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00ce60 con 0x7fec741488f0 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2026-02-22T23:29:03.448 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 monclient: dump: 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 monclient: _finish_auth 0 2026-02-22T23:29:03.449 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 monclient: _check_auth_tickets 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 1 -- 10.20.193.6:0/1171937007 <== mon.0 v2:10.20.193.6:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5c004cf0 con 0x7fec741488f0 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec79790640 5 monclient: authenticate success, global_id 5310 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec79790640 10 monclient: _renew_subs 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec79790640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec740737b0 con 0x7fec741488f0 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.428+0000 7fec537fe640 10 monclient: handle_config config(0 keys) 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec537fe640 1 -- 10.20.193.6:0/1171937007 <== mon.0 v2:10.20.193.6:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c007350 con 0x7fec741488f0 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec537fe640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec537fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec537fe640 10 monclient: dump: 2026-02-22T23:29:03.450 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec717fa640 4 set_mon_vals no callback set 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 10 client.5310.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 10 monclient: _renew_subs 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:29:03.451 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec74073d70 con 0x7fec741488f0 2026-02-22T23:29:03.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 1 librados: init done 2026-02-22T23:29:03.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:29:03.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:29:03.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 10 monclient: _send_mon_message to mon.a at v2:10.20.193.6:3300/0 2026-02-22T23:29:03.452 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.429+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec74159ed0 con 0x7fec741488f0 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.441+0000 7fec537fe640 1 -- 10.20.193.6:0/1171937007 <== mon.0 v2:10.20.193.6:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7fec5c0074f0 con 0x7fec741488f0 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.441+0000 7fec537fe640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fec40078e50 0x7fec4007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.441+0000 7fec72ffd640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fec40078e50 0x7fec4007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.441+0000 7fec72ffd640 10 monclient: get_auth_request con 0x7fec40078e50 auth_method 0 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.443+0000 7fec72ffd640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fec40078e50 0x7fec4007b310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fec68036000 tx=0x7fec6803b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.443+0000 7fec537fe640 10 client.5310.objecter ms_handle_connect 0x7fec40078e50 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.443+0000 7fec537fe640 1 -- 10.20.193.6:0/1171937007 <== mon.0 v2:10.20.193.6:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7fec5c05df40 con 0x7fec741488f0 2026-02-22T23:29:03.453 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.443+0000 7fec537fe640 10 client.5310.objecter ms_dispatch2 0x7fec74000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:29:03.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.443+0000 7fec537fe640 3 client.5310.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:29:03.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.443+0000 7fec537fe640 3 client.5310.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:29:03.454 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.443+0000 7fec537fe640 20 client.5310.objecter dump_active .. 0 homeless 2026-02-22T23:29:03.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.541+0000 7fec537fe640 1 -- 10.20.193.6:0/1171937007 <== mon.0 v2:10.20.193.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7fec741f4880 con 0x7fec741488f0 2026-02-22T23:29:03.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.541+0000 7fec537fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:29:03.543 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.541+0000 7fec537fe640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:29:03.624 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.623+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 --> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec741578e0 con 0x7fec40078e50 2026-02-22T23:29:03.626 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.623+0000 7fec537fe640 1 -- 10.20.193.6:0/1171937007 <== mgr.4098 v2:10.20.193.189:6800/3784423838 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+45306 (secure 0 0 0) 0x7fec741578e0 con 0x7fec40078e50 2026-02-22T23:29:03.626 INFO:teuthology.orchestra.run.trial006.stdout: 2026-02-22T23:29:03.628 INFO:teuthology.orchestra.run.trial006.stderr:dumped all 2026-02-22T23:29:03.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 10 librados: watch_flush enter 2026-02-22T23:29:03.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 10 librados: watch_flush exit 2026-02-22T23:29:03.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 20 client.5310.objecter shutdown clearing up homeless session... 2026-02-22T23:29:03.628 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 10 client.5310.objecter successfully canceled tick 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fec40078e50 msgr2=0x7fec4007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fec40078e50 0x7fec4007b310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fec68036000 tx=0x7fec6803b040 comp rx=0 tx=0).stop 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 10 monclient: shutdown 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec741488f0 msgr2=0x7fec741f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec741488f0 0x7fec741f51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00c730 tx=0x7fec5c00cc00 comp rx=0 tx=0).stop 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 shutdown_connections 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7fec40078e50 0x7fec4007b310 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7fec74148d70 0x7fec741f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7fec741488f0 0x7fec741f51b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.629 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 --2- 10.20.193.6:0/1171937007 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7fec60003dd0 0x7fec741f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 >> 10.20.193.6:0/1171937007 conn(0x7fec740cd780 msgr2=0x7fec74158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:29:03.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 shutdown_connections 2026-02-22T23:29:03.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 -- 10.20.193.6:0/1171937007 wait complete. 2026-02-22T23:29:03.630 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.624+0000 7fec79790640 1 librados: shutdown 2026-02-22T23:29:03.635 INFO:teuthology.orchestra.run.trial006.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2026-02-22T23:29:01.812152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":13,"kb":1509949440,"kb_used":436980,"kb_used_data":5592,"kb_used_omap":24,"kb_used_meta":431079,"kb_avail":1509512460,"statfs":{"total":1546188226560,"available":1545740759040,"internally_reserved":0,"allocated":5726208,"data_stored":2779510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25502,"internal_metadata":441424994},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001820"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:43.292016+0000","last_change":"2026-02-22T23:28:43.294704+0000","last_active":"2026-02-22T23:28:43.292016+0000","last_peered":"2026-02-22T23:28:43.292016+0000","last_clean":"2026-02-22T23:28:43.292016+0000","last_became_active":"2026-02-22T23:23:24.999148+0000","last_became_peered":"2026-02-22T23:23:24.999148+0000","last_unstale":"2026-02-22T23:28:43.292016+0000","last_undegraded":"2026-02-22T23:28:43.292016+0000","last_fullsized":"2026-02-22T23:28:43.292016+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:28:43.289727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:28:43.289727+0000","last_clean_scrub_stamp":"2026-02-22T23:28:43.289727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:11:47.649823+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0.033743088999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,0],"acting":[15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:41.948925+0000","last_change":"2026-02-22T23:28:41.953863+0000","last_active":"2026-02-22T23:28:41.948925+0000","last_peered":"2026-02-22T23:28:41.948925+0000","last_clean":"2026-02-22T23:28:41.948925+0000","last_became_active":"2026-02-22T23:23:24.966693+0000","last_became_peered":"2026-02-22T23:23:24.966693+0000","last_unstale":"2026-02-22T23:28:41.948925+0000","last_undegraded":"2026-02-22T23:28:41.948925+0000","last_fullsized":"2026-02-22T23:28:41.948925+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:28:41.945836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:28:41.945836+0000","last_clean_scrub_stamp":"2026-02-22T23:28:41.945836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:08:35.733844+0000","scrub_duration":190,"objects_trimmed":0,"snaptrim_duration":0.033152011000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13],"acting":[14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:40.269522+0000","last_change":"2026-02-22T23:28:40.274352+0000","last_active":"2026-02-22T23:28:40.269522+0000","last_peered":"2026-02-22T23:28:40.269522+0000","last_clean":"2026-02-22T23:28:40.269522+0000","last_became_active":"2026-02-22T23:23:24.806260+0000","last_became_peered":"2026-02-22T23:23:24.806260+0000","last_unstale":"2026-02-22T23:28:40.269522+0000","last_undegraded":"2026-02-22T23:28:40.269522+0000","last_fullsized":"2026-02-22T23:28:40.269522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:28:40.266273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:28:40.266273+0000","last_clean_scrub_stamp":"2026-02-22T23:28:40.266273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:33:19.017927+0000","scrub_duration":204,"objects_trimmed":0,"snaptrim_duration":0.032290068999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2],"acting":[9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:40.579480+0000","last_change":"2026-02-22T23:28:40.584202+0000","last_active":"2026-02-22T23:28:40.579480+0000","last_peered":"2026-02-22T23:28:40.579480+0000","last_clean":"2026-02-22T23:28:40.579480+0000","last_became_active":"2026-02-22T23:23:24.706162+0000","last_became_peered":"2026-02-22T23:23:24.706162+0000","last_unstale":"2026-02-22T23:28:40.579480+0000","last_undegraded":"2026-02-22T23:28:40.579480+0000","last_fullsized":"2026-02-22T23:28:40.579480+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:28:40.576367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:28:40.576367+0000","last_clean_scrub_stamp":"2026-02-22T23:28:40.576367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:00:03.621842+0000","scrub_duration":195,"objects_trimmed":0,"snaptrim_duration":0.041859805999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"28'3","reported_seq":35,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:42.994205+0000","last_change":"2026-02-22T23:28:42.995047+0000","last_active":"2026-02-22T23:28:42.994205+0000","last_peered":"2026-02-22T23:28:42.994205+0000","last_clean":"2026-02-22T23:28:42.994205+0000","last_became_active":"2026-02-22T23:23:25.054022+0000","last_became_peered":"2026-02-22T23:23:25.054022+0000","last_unstale":"2026-02-22T23:28:42.994205+0000","last_undegraded":"2026-02-22T23:28:42.994205+0000","last_fullsized":"2026-02-22T23:28:42.994205+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'3","last_scrub_stamp":"2026-02-22T23:28:42.993466+0000","last_deep_scrub":"28'3","last_deep_scrub_stamp":"2026-02-22T23:28:42.993466+0000","last_clean_scrub_stamp":"2026-02-22T23:28:42.993466+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:29:08.152862+0000","scrub_duration":171,"objects_trimmed":0,"snaptrim_duration":0.070943361999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10],"acting":[3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:38.830316+0000","last_change":"2026-02-22T23:28:38.830676+0000","last_active":"2026-02-22T23:28:38.830316+0000","last_peered":"2026-02-22T23:28:38.830316+0000","last_clean":"2026-02-22T23:28:38.830316+0000","last_became_active":"2026-02-22T23:23:24.462347+0000","last_became_peered":"2026-02-22T23:23:24.462347+0000","last_unstale":"2026-02-22T23:28:38.830316+0000","last_undegraded":"2026-02-22T23:28:38.830316+0000","last_fullsized":"2026-02-22T23:28:38.830316+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:28:38.829527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:28:38.829527+0000","last_clean_scrub_stamp":"2026-02-22T23:28:38.829527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T11:09:16.436294+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.033702211000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4],"acting":[8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:41.926184+0000","last_change":"2026-02-22T23:28:41.930871+0000","last_active":"2026-02-22T23:28:41.926184+0000","last_peered":"2026-02-22T23:28:41.926184+0000","last_clean":"2026-02-22T23:28:41.926184+0000","last_became_active":"2026-02-22T23:23:24.464964+0000","last_became_peered":"2026-02-22T23:23:24.464964+0000","last_unstale":"2026-02-22T23:28:41.926184+0000","last_undegraded":"2026-02-22T23:28:41.926184+0000","last_fullsized":"2026-02-22T23:28:41.926184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:28:41.922930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:28:41.922930+0000","last_clean_scrub_stamp":"2026-02-22T23:28:41.922930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:10:52.098908+0000","scrub_duration":117,"objects_trimmed":0,"snaptrim_duration":0.055238717,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'2","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:37.614684+0000","last_change":"2026-02-22T23:28:37.619874+0000","last_active":"2026-02-22T23:28:37.614684+0000","last_peered":"2026-02-22T23:28:37.614684+0000","last_clean":"2026-02-22T23:28:37.614684+0000","last_became_active":"2026-02-22T23:23:24.503509+0000","last_became_peered":"2026-02-22T23:23:24.503509+0000","last_unstale":"2026-02-22T23:28:37.614684+0000","last_undegraded":"2026-02-22T23:28:37.614684+0000","last_fullsized":"2026-02-22T23:28:37.614684+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'2","last_scrub_stamp":"2026-02-22T23:28:37.612568+0000","last_deep_scrub":"28'2","last_deep_scrub_stamp":"2026-02-22T23:28:37.612568+0000","last_clean_scrub_stamp":"2026-02-22T23:28:37.612568+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:07:19.199248+0000","scrub_duration":270,"objects_trimmed":0,"snaptrim_duration":0.032869134000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,15],"acting":[4,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"28'34","reported_seq":77,"reported_epoch":28,"state":"active+clean","last_fresh":"2026-02-22T23:28:38.120813+0000","last_change":"2026-02-22T23:28:38.126518+0000","last_active":"2026-02-22T23:28:38.120813+0000","last_peered":"2026-02-22T23:28:38.120813+0000","last_clean":"2026-02-22T23:28:38.120813+0000","last_became_active":"2026-02-22T23:23:22.740754+0000","last_became_peered":"2026-02-22T23:23:22.740754+0000","last_unstale":"2026-02-22T23:28:38.120813+0000","last_undegraded":"2026-02-22T23:28:38.120813+0000","last_fullsized":"2026-02-22T23:28:38.120813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"28'34","last_scrub_stamp":"2026-02-22T23:28:38.117922+0000","last_deep_scrub":"28'34","last_deep_scrub_stamp":"2026-02-22T23:28:38.117922+0000","last_clean_scrub_stamp":"2026-02-22T23:28:38.117922+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:51:17.798315+0000","scrub_duration":281,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5],"acting":[8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":13},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":23,"seq":98784247879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,5,6,7,9,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":15,"up_from":22,"seq":94489280583,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,2,4,5,6,7,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 15 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":13,"up_from":22,"seq":94489280584,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[1,2,3,4,5,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 13 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":21,"seq":90194313286,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,7,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 2 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":6,"up_from":21,"seq":90194313287,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,4,5,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 6 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":7,"up_from":21,"seq":90194313287,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27589067},"hb_peers":[2,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 7 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":9,"up_from":21,"seq":90194313286,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[2,4,6,7,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 9 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":10,"up_from":21,"seq":90194313286,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27589057},"hb_peers":[1,4,6,7,8,9,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 10 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":11,"up_from":21,"seq":90194313287,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27589061},"hb_peers":[0,4,6,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 11 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":14,"up_from":21,"seq":90194313287,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,4,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 14 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":1,"up_from":22,"seq":94489280583,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2,4,5,6,7,8,9,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":3,"up_from":22,"seq":94489280583,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":311296,"data_stored":124006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27589064},"hb_peers":[2,4,5,7,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280583,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,3,5,7,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 4 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":5,"up_from":22,"seq":94489280582,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27589063},"hb_peers":[0,4,6,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":22,"seq":94489280583,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":708608,"data_stored":521827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[3,4,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 8 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":12,"up_from":22,"seq":94489280583,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27264,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":307200,"data_stored":123987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2,3,4,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 12 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:29:03.638 DEBUG:teuthology.orchestra.run.trial006:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-22T23:29:03.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.702+0000 7f5bc002b640 10 monclient: get_monmap_and_config 2026-02-22T23:29:03.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.702+0000 7f5bc002b640 10 monclient: build_initial_monmap 2026-02-22T23:29:03.705 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.702+0000 7f5bc002b640 10 monclient: monmap: 2026-02-22T23:29:03.705 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:29:03.705 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.705 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:29:03.704540+0000 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:29:03.704540+0000 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:0: v1:10.20.193.6:6789/0 mon.noname-a-legacy 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:1: v1:10.20.193.40:6789/0 mon.noname-b-legacy 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:2: v1:10.20.193.80:6789/0 mon.noname-c-legacy 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:3: v2:10.20.193.6:3300/0 mon.noname-a 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:4: v2:10.20.193.40:3300/0 mon.noname-b 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr:5: v2:10.20.193.80:3300/0 mon.noname-c 2026-02-22T23:29:03.706 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 1 Processor -- start 2026-02-22T23:29:03.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 1 -- start start 2026-02-22T23:29:03.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 10 monclient: init 2026-02-22T23:29:03.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 10 monclient: _reopen_session rank -1 2026-02-22T23:29:03.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 10 monclient: _add_conns ranks=[1,4,2,0,3,5] 2026-02-22T23:29:03.709 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 10 monclient(hunting): picked mon.noname-b-legacy con 0x7f5bb806ff20 addr v1:10.20.193.40:6789/0 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f5bb80c99a0 0x7f5bb8070300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 10 monclient(hunting): picked mon.noname-b con 0x7f5bb80c99a0 addr v2:10.20.193.40:3300/0 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.703+0000 7f5bc002b640 10 monclient(hunting): picked mon.noname-c-legacy con 0x7f5bb815a5e0 addr v1:10.20.193.80:6789/0 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bc002b640 1 -- --> v1:10.20.193.40:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb80c7e90 con 0x7f5bb806ff20 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bc002b640 1 -- --> v1:10.20.193.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb80c8170 con 0x7f5bb815a5e0 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bc002b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bc002b640 1 -- --> v2:10.20.193.40:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb8159a40 con 0x7f5bb80c99a0 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bc002b640 10 monclient(hunting): _renew_subs 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bc002b640 10 monclient(hunting): authenticate will time out at 1418.432024s 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bbd59f640 1 --2- >> v2:10.20.193.40:3300/0 conn(0x7f5bb80c99a0 0x7f5bb8070300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bbe5a1640 1 --1- >> v1:10.20.193.80:6789/0 conn(0x7f5bb815a5e0 0x7f5bb814fe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:10.20.193.80:6789/0 says I am v1:10.20.193.6:48568/0 (socket says 10.20.193.6:48568) 2026-02-22T23:29:03.710 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.704+0000 7f5bbe5a1640 1 -- 10.20.193.6:0/2180902950 learned_addr learned my addr 10.20.193.6:0/2180902950 (peer_addr_for_me v1:10.20.193.6:0/0) 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): get_auth_request con 0x7f5bb80c99a0 auth_method 0 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbcd9e640 1 -- 10.20.193.6:0/2180902950 <== mon.2 v1:10.20.193.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969291063 0 0) 0x7f5bb80c8170 con 0x7f5bb815a5e0 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbcd9e640 10 monclient(hunting): _negotiate 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbcd9e640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbcd9e640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.711 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbcd9e640 10 monclient(hunting): my global_id is 5444 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbcd9e640 1 -- 10.20.193.6:0/2180902950 --> v1:10.20.193.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ba0003620 con 0x7f5bb815a5e0 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient(hunting): handle_auth_done global_id 5293 payload 274 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 1 -- 10.20.193.6:0/2180902950 >> v1:10.20.193.80:6789/0 conn(0x7f5bb815a5e0 legacy=0x7f5bb814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 1 -- 10.20.193.6:0/2180902950 >> v1:10.20.193.40:6789/0 conn(0x7f5bb806ff20 legacy=0x7f5bb8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient: _finish_hunting 0 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 1 monclient: found mon.noname-b 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 10 monclient: _send_mon_message to mon.noname-b at v2:10.20.193.40:3300/0 2026-02-22T23:29:03.712 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.706+0000 7f5bbd59f640 1 -- 10.20.193.6:0/2180902950 --> v2:10.20.193.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb80c8450 con 0x7f5bb80c99a0 2026-02-22T23:29:03.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.707+0000 7f5bbd59f640 1 --2- 10.20.193.6:0/2180902950 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb8070300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0098d0 tx=0x7f5bac02eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f760e7981be97c server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:29:03.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 1 -- 10.20.193.6:0/2180902950 <== mon.1 v2:10.20.193.40:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac03c070 con 0x7f5bb80c99a0 2026-02-22T23:29:03.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.717 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: dump: 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.718 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: _finish_auth 0 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: _check_auth_tickets 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 1 -- 10.20.193.6:0/2180902950 <== mon.1 v2:10.20.193.40:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bac02fba0 con 0x7f5bb80c99a0 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 5 monclient: authenticate success, global_id 5293 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 20 monclient: get_monmap_and_config waiting for monmap|config 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: handle_config config(0 keys) 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 1 -- 10.20.193.6:0/2180902950 <== mon.1 v2:10.20.193.40:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac037440 con 0x7f5bb80c99a0 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 10 monclient: get_monmap_and_config success 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 4 set_mon_vals no callback set 2026-02-22T23:29:03.719 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: got monmap 1 from mon.b (according to old e1) 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bbcd9e640 10 monclient: dump: 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.720 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 10 monclient: shutdown 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 1 -- 10.20.193.6:0/2180902950 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 msgr2=0x7f5bb8070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 1 --2- 10.20.193.6:0/2180902950 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb8070300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0098d0 tx=0x7f5bac02eed0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 1 -- 10.20.193.6:0/2180902950 shutdown_connections 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 1 --2- 10.20.193.6:0/2180902950 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb8070300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 1 -- 10.20.193.6:0/2180902950 >> 10.20.193.6:0/2180902950 conn(0x7f5bb80cd780 msgr2=0x7f5bb80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 1 -- 10.20.193.6:0/2180902950 shutdown_connections 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.709+0000 7f5bc002b640 1 -- 10.20.193.6:0/2180902950 wait complete. 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 10 monclient: build_initial_monmap 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 10 monclient: monmap: 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:epoch 0 2026-02-22T23:29:03.721 INFO:teuthology.orchestra.run.trial006.stderr:fsid 00000000-0000-0000-0000-000000000000 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 0.000000 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:created 0.000000 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 0 (unknown) 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.noname-a 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.noname-b 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.noname-c 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 1 librados: starting msgr at 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 1 librados: starting objecter 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 1 Processor -- start 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 1 -- start start 2026-02-22T23:29:03.722 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 1 librados: setting wanted keys 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 1 librados: calling monclient init 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.710+0000 7f5bc002b640 10 monclient: init 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient: _reopen_session rank -1 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient: _add_conns ranks=[2,1,0] 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5bb806ff20 0x7f5bb814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): picked mon.noname-c con 0x7f5bb806ff20 addr [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb81f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): picked mon.noname-b con 0x7f5bb80c99a0 addr [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5bb815a5e0 0x7f5bb81f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): picked mon.noname-a con 0x7f5bb815a5e0 addr [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.723 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 1 -- --> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8150520 con 0x7f5bb815a5e0 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 1 -- --> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb80c8a10 con 0x7f5bb80c99a0 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): start opening mon connection 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 1 -- --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81506a0 con 0x7f5bb806ff20 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): _renew_subs 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bc002b640 10 monclient(hunting): authenticate will time out at 1418.439163s 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbdda0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5bb806ff20 0x7f5bb814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbe5a1640 1 --2- >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5bb815a5e0 0x7f5bb81f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbd59f640 1 --2- >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb81f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbdda0640 1 --2- >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5bb806ff20 0x7f5bb814bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.80:3300/0 says I am v2:10.20.193.6:45842/0 (socket says 10.20.193.6:45842) 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbdda0640 1 -- 10.20.193.6:0/1454311385 learned_addr learned my addr 10.20.193.6:0/1454311385 (peer_addr_for_me v2:10.20.193.6:0/0) 2026-02-22T23:29:03.724 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbdda0640 10 monclient(hunting): get_auth_request con 0x7f5bb806ff20 auth_method 0 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbdda0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbdda0640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbdda0640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbd59f640 10 monclient(hunting): get_auth_request con 0x7f5bb80c99a0 auth_method 0 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbd59f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbd59f640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbd59f640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbe5a1640 10 monclient(hunting): get_auth_request con 0x7f5bb815a5e0 auth_method 0 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbe5a1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbe5a1640 10 monclient(hunting): _init_auth method 2 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.711+0000 7f5bbe5a1640 10 monclient(hunting): _init_auth creating new auth 2026-02-22T23:29:03.725 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbdda0640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbdda0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbdda0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbd59f640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbd59f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbd59f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbe5a1640 10 monclient(hunting): handle_auth_reply_more payload 9 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbe5a1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.712+0000 7f5bbe5a1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 10 monclient(hunting): handle_auth_done global_id 5447 payload 824 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 1 -- 10.20.193.6:0/1454311385 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 msgr2=0x7f5bb81f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb81f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 1 -- 10.20.193.6:0/1454311385 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5bb815a5e0 msgr2=0x7f5bb81f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:03.726 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5bb815a5e0 0x7f5bb81f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 10 monclient: _finish_hunting 0 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 1 monclient: found mon.noname-c 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 20 monclient: _un_backoff reopen_interval_multipler now 1 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 10 monclient: _send_mon_message to mon.noname-c at v2:10.20.193.80:3300/0 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbdda0640 1 -- 10.20.193.6:0/1454311385 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb814ca80 con 0x7f5bb806ff20 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.713+0000 7f5bbd59f640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb81f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.714+0000 7f5bbdda0640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5bb806ff20 0x7f5bb814bf00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800c730 tx=0x7f5ba800cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.714+0000 7f5ba67fc640 10 client.?.objecter ms_handle_connect 0x7f5bb806ff20 2026-02-22T23:29:03.727 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.714+0000 7f5ba67fc640 10 client.?.objecter resend_mon_ops 2026-02-22T23:29:03.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80052d0 con 0x7f5bb806ff20 2026-02-22T23:29:03.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2026-02-22T23:29:03.729 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: dump: 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.730 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: _finish_auth 0 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: _check_auth_tickets 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba8005470 con 0x7f5bb806ff20 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 5 monclient: authenticate success, global_id 5447 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 10 monclient: _renew_subs 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb81f5380 con 0x7f5bb806ff20 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: handle_config config(0 keys) 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80038d0 con 0x7f5bb806ff20 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: handle_monmap mon_map magic: 0 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba7fff640 4 set_mon_vals no callback set 2026-02-22T23:29:03.731 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5ba67fc640 10 monclient: dump: 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:epoch 1 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:fsid f24f9547-07c5-4c58-99d4-b3bf9b186dd1 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:last_changed 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:created 2026-02-22T23:19:53.768742+0000 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:min_mon_release 19 (squid) 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:election_strategy: 1 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:0: [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] mon.a 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:1: [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] mon.b 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:2: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.c 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 10 client.5447.objecter _maybe_request_map subscribing (onetime) to next osd map 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 10 monclient: _renew_subs 2026-02-22T23:29:03.732 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:29:03.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb81f5940 con 0x7f5bb806ff20 2026-02-22T23:29:03.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 1 librados: init done 2026-02-22T23:29:03.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2026-02-22T23:29:03.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.716+0000 7f5bc002b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:29:03.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.717+0000 7f5bc002b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:29:03.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.717+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb8070300 con 0x7f5bb806ff20 2026-02-22T23:29:03.733 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.727+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 4 ==== mgrmap(e 5) ==== 99901+0+0 (secure 0 0 0) 0x7f5ba8003a70 con 0x7f5bb806ff20 2026-02-22T23:29:03.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5ba67fc640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b88078e50 0x7f5b8807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-22T23:29:03.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5bbd59f640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b88078e50 0x7f5b8807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-22T23:29:03.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5bbd59f640 10 monclient: get_auth_request con 0x7f5b88078e50 auth_method 0 2026-02-22T23:29:03.734 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 8944+0+0 (secure 0 0 0) 0x7f5ba805e0a0 con 0x7f5bb806ff20 2026-02-22T23:29:03.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5ba67fc640 10 client.5447.objecter ms_dispatch2 0x7f5bb8000b90 osd_map(28..28 src has 1..28) 2026-02-22T23:29:03.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5ba67fc640 3 client.5447.objecter handle_osd_map got epochs [28,28] > 0 2026-02-22T23:29:03.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5ba67fc640 3 client.5447.objecter handle_osd_map decoding full epoch 28 2026-02-22T23:29:03.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5ba67fc640 20 client.5447.objecter dump_active .. 0 homeless 2026-02-22T23:29:03.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5ba67fc640 10 client.5447.objecter ms_handle_connect 0x7f5b88078e50 2026-02-22T23:29:03.735 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.728+0000 7f5bbd59f640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b88078e50 0x7f5b8807b310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5bac039000 tx=0x7f5bac033000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-22T23:29:03.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.835+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195034 (secure 0 0 0) 0x7f5ba8062060 con 0x7f5bb806ff20 2026-02-22T23:29:03.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.835+0000 7f5ba67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2026-02-22T23:29:03.836 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.835+0000 7f5ba67fc640 10 monclient: _finish_command 1 = system:0 2026-02-22T23:29:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.918+0000 7f5bc002b640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-22T23:29:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.918+0000 7f5bc002b640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-22T23:29:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.918+0000 7f5bc002b640 10 monclient: _send_mon_message to mon.c at v2:10.20.193.80:3300/0 2026-02-22T23:29:03.919 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:03.918+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 --> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5bb806c010 con 0x7f5bb806ff20 2026-02-22T23:29:04.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.037+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 7 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5ba8066360 con 0x7f5bb806ff20 2026-02-22T23:29:04.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.037+0000 7f5ba67fc640 10 monclient: handle_config config(1 keys) 2026-02-22T23:29:04.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.038+0000 7f5ba7fff640 4 set_mon_vals no callback set 2026-02-22T23:29:04.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.038+0000 7f5ba7fff640 10 set_mon_vals mon_health_to_clog = false 2026-02-22T23:29:04.039 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.038+0000 7f5ba67fc640 1 -- 10.20.193.6:0/1454311385 <== mon.2 v2:10.20.193.80:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) ==== 123+0+0 (secure 0 0 0) 0x7f5ba806b210 con 0x7f5bb806ff20 2026-02-22T23:29:04.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.038+0000 7f5ba67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2026-02-22T23:29:04.040 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.038+0000 7f5ba67fc640 10 monclient: _finish_command 2 = system:0 2026-02-22T23:29:04.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 10 librados: watch_flush enter 2026-02-22T23:29:04.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 10 librados: watch_flush exit 2026-02-22T23:29:04.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 20 client.5447.objecter shutdown clearing up homeless session... 2026-02-22T23:29:04.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 10 client.5447.objecter successfully canceled tick 2026-02-22T23:29:04.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b88078e50 msgr2=0x7f5b8807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:04.042 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b88078e50 0x7f5b8807b310 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5bac039000 tx=0x7f5bac033000 comp rx=0 tx=0).stop 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 10 monclient: shutdown 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 20 monclient: shutdown discarding 0 pending message(s) 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5bb806ff20 msgr2=0x7f5bb814bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5bb806ff20 0x7f5bb814bf00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800c730 tx=0x7f5ba800cc00 comp rx=0 tx=0).stop 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 shutdown_connections 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.189:6800/3784423838,v1:10.20.193.189:6801/3784423838] conn(0x7f5b88078e50 0x7f5b8807b310 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.6:3300/0,v1:10.20.193.6:6789/0] conn(0x7f5bb815a5e0 0x7f5bb81f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.40:3300/0,v1:10.20.193.40:6789/0] conn(0x7f5bb80c99a0 0x7f5bb81f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 --2- 10.20.193.6:0/1454311385 >> [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] conn(0x7f5bb806ff20 0x7f5bb814bf00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 >> 10.20.193.6:0/1454311385 conn(0x7f5bb80cd780 msgr2=0x7f5bb80cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 shutdown_connections 2026-02-22T23:29:04.043 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 -- 10.20.193.6:0/1454311385 wait complete. 2026-02-22T23:29:04.044 INFO:teuthology.orchestra.run.trial006.stderr:2026-02-22T23:29:04.040+0000 7f5bc002b640 1 librados: shutdown 2026-02-22T23:29:04.050 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy 2026-02-22T23:29:04.053 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-22T23:29:04.054 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-22T23:29:04.054 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-22T23:29:04.054 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:04.062 INFO:tasks.ceph.osd.0.trial006.stderr:2026-02-22T23:29:04.055+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 45040) UID: 0 2026-02-22T23:29:04.062 INFO:tasks.ceph.osd.0.trial006.stderr:2026-02-22T23:29:04.056+0000 beae640 -1 osd.0 28 *** Got signal Terminated *** 2026-02-22T23:29:10.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-22T23:29:10.156 INFO:tasks.ceph.osd.0:Stopped 2026-02-22T23:29:10.157 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-22T23:29:10.157 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:10.164 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:29:10.158+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 45043) UID: 0 2026-02-22T23:29:10.165 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:29:10.159+0000 beae640 -1 osd.4 30 *** Got signal Terminated *** 2026-02-22T23:29:11.890 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:29:11.888+0000 a699640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-22T23:29:11.892 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:29:11.890+0000 c6af640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-22T23:29:11.892 INFO:tasks.ceph.osd.4.trial006.stderr:2026-02-22T23:29:11.891+0000 9e98640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-22T23:29:13.458 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:57.708 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.461 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:57.711 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.471 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:07:57.721 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.487 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:57.698 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.487 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:57.733 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.488 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:57.699 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.489 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:57.735 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.498 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:07:57.745 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:13.498 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:07:57.710 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:16.259 INFO:tasks.ceph.osd.4:Stopped 2026-02-22T23:29:16.260 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2026-02-22T23:29:16.260 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:16.268 INFO:tasks.ceph.osd.8.trial006.stderr:2026-02-22T23:29:16.261+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.8.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 8 (PID: 45045) UID: 0 2026-02-22T23:29:16.268 INFO:tasks.ceph.osd.8.trial006.stderr:2026-02-22T23:29:16.262+0000 beae640 -1 osd.8 32 *** Got signal Terminated *** 2026-02-22T23:29:17.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2026-02-22T23:29:17.442 INFO:tasks.ceph.osd.8.trial006.stderr:2026-02-22T23:29:17.439+0000 c6af640 -1 monclient: get_auth_request but no auth handler is set up 2026-02-22T23:29:22.362 INFO:tasks.ceph.osd.8:Stopped 2026-02-22T23:29:22.362 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2026-02-22T23:29:22.363 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:22.370 INFO:tasks.ceph.osd.12.trial006.stderr:2026-02-22T23:29:22.364+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.12.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 12 (PID: 45048) UID: 0 2026-02-22T23:29:22.370 INFO:tasks.ceph.osd.12.trial006.stderr:2026-02-22T23:29:22.365+0000 beae640 -1 osd.12 34 *** Got signal Terminated *** 2026-02-22T23:29:23.847 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.8 is failed for ~0s 2026-02-22T23:29:28.464 INFO:tasks.ceph.osd.12:Stopped 2026-02-22T23:29:28.464 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-22T23:29:28.465 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:28.471 INFO:tasks.ceph.osd.1.trial040.stderr:2026-02-22T23:29:28.463+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 44601) UID: 0 2026-02-22T23:29:28.473 INFO:tasks.ceph.osd.1.trial040.stderr:2026-02-22T23:29:28.463+0000 beae640 -1 osd.1 35 *** Got signal Terminated *** 2026-02-22T23:29:30.563 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.8 has been restored 2026-02-22T23:29:33.486 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:17.736 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.488 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:17.738 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.498 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:17.748 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.514 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:17.725 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.514 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:17.761 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.515 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:17.727 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.516 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:17.763 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.526 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:17.737 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:33.527 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:17.773 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:34.566 INFO:tasks.ceph.osd.1:Stopped 2026-02-22T23:29:34.566 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-22T23:29:34.567 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:34.573 INFO:tasks.ceph.osd.5.trial040.stderr:2026-02-22T23:29:34.565+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 44604) UID: 0 2026-02-22T23:29:34.574 INFO:tasks.ceph.osd.5.trial040.stderr:2026-02-22T23:29:34.565+0000 beae640 -1 osd.5 37 *** Got signal Terminated *** 2026-02-22T23:29:40.668 INFO:tasks.ceph.osd.5:Stopped 2026-02-22T23:29:40.668 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2026-02-22T23:29:40.668 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:40.676 INFO:tasks.ceph.osd.9.trial040.stderr:2026-02-22T23:29:40.667+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.9.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 9 (PID: 44607) UID: 0 2026-02-22T23:29:40.677 INFO:tasks.ceph.osd.9.trial040.stderr:2026-02-22T23:29:40.668+0000 beae640 -1 osd.9 39 *** Got signal Terminated *** 2026-02-22T23:29:46.771 INFO:tasks.ceph.osd.9:Stopped 2026-02-22T23:29:46.771 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2026-02-22T23:29:46.771 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:46.778 INFO:tasks.ceph.osd.13.trial040.stderr:2026-02-22T23:29:46.769+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.13.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 13 (PID: 44636) UID: 0 2026-02-22T23:29:46.780 INFO:tasks.ceph.osd.13.trial040.stderr:2026-02-22T23:29:46.770+0000 beae640 -1 osd.13 41 *** Got signal Terminated *** 2026-02-22T23:29:49.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.13 is failed for ~0s 2026-02-22T23:29:52.873 INFO:tasks.ceph.osd.13:Stopped 2026-02-22T23:29:52.874 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-22T23:29:52.874 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:52.881 INFO:tasks.ceph.osd.2.trial080.stderr:2026-02-22T23:29:52.875+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 44702) UID: 0 2026-02-22T23:29:52.882 INFO:tasks.ceph.osd.2.trial080.stderr:2026-02-22T23:29:52.875+0000 beae640 -1 osd.2 43 *** Got signal Terminated *** 2026-02-22T23:29:53.530 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:37.780 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.532 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:37.783 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.545 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:37.795 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.556 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:37.803 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.559 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:37.806 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.561 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:37.772 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.562 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:37.773 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.571 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:37.817 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:53.574 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:37.785 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:29:56.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-22T23:29:56.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.13 has been restored 2026-02-22T23:29:58.975 INFO:tasks.ceph.osd.2:Stopped 2026-02-22T23:29:58.975 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2026-02-22T23:29:58.975 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:29:58.983 INFO:tasks.ceph.osd.6.trial080.stderr:2026-02-22T23:29:58.977+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 44703) UID: 0 2026-02-22T23:29:58.983 INFO:tasks.ceph.osd.6.trial080.stderr:2026-02-22T23:29:58.977+0000 beae640 -1 osd.6 44 *** Got signal Terminated *** 2026-02-22T23:30:02.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-22T23:30:08.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2026-02-22T23:30:11.178 INFO:tasks.ceph.osd.6:Stopped 2026-02-22T23:30:11.178 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2026-02-22T23:30:11.179 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:11.186 INFO:tasks.ceph.osd.10.trial080.stderr:2026-02-22T23:30:11.180+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.10.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 10 (PID: 44705) UID: 0 2026-02-22T23:30:11.186 INFO:tasks.ceph.osd.10.trial080.stderr:2026-02-22T23:30:11.181+0000 beae640 -1 osd.10 45 *** Got signal Terminated *** 2026-02-22T23:30:13.557 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:57.807 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.559 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:57.809 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.569 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:08:57.819 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.584 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:57.830 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.585 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:57.832 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.588 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:57.800 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.589 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:57.801 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.595 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:08:57.841 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:13.599 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:08:57.811 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:14.537 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.10 is failed for ~0s 2026-02-22T23:30:14.537 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2026-02-22T23:30:17.280 INFO:tasks.ceph.osd.10:Stopped 2026-02-22T23:30:17.280 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2026-02-22T23:30:17.280 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:17.288 INFO:tasks.ceph.osd.14.trial080.stderr:2026-02-22T23:30:17.282+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.14.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 14 (PID: 44709) UID: 0 2026-02-22T23:30:17.288 INFO:tasks.ceph.osd.14.trial080.stderr:2026-02-22T23:30:17.283+0000 beae640 -1 osd.14 47 *** Got signal Terminated *** 2026-02-22T23:30:20.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.14 is failed for ~0s 2026-02-22T23:30:20.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.10 has been restored 2026-02-22T23:30:23.382 INFO:tasks.ceph.osd.14:Stopped 2026-02-22T23:30:23.383 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-22T23:30:23.383 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:23.401 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-22T23:30:23.397+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 44895) UID: 0 2026-02-22T23:30:23.402 INFO:tasks.ceph.osd.3.trial189.stderr:2026-02-22T23:30:23.398+0000 beae640 -1 osd.3 48 *** Got signal Terminated *** 2026-02-22T23:30:26.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2026-02-22T23:30:26.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.14 has been restored 2026-02-22T23:30:29.484 INFO:tasks.ceph.osd.3:Stopped 2026-02-22T23:30:29.485 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2026-02-22T23:30:29.485 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:29.497 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-22T23:30:29.492+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 44897) UID: 0 2026-02-22T23:30:29.497 INFO:tasks.ceph.osd.7.trial189.stderr:2026-02-22T23:30:29.493+0000 beae640 -1 osd.7 48 *** Got signal Terminated *** 2026-02-22T23:30:32.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2026-02-22T23:30:33.566 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:09:17.816 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.591 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:09:17.802 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.592 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:09:17.803 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.593 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:09:17.839 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.594 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:09:17.844 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.595 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:09:17.842 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.601 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:09:17.812 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.604 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:09:17.850 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:33.604 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:09:17.854 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:35.587 INFO:tasks.ceph.osd.7:Stopped 2026-02-22T23:30:35.588 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2026-02-22T23:30:35.588 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:35.595 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-22T23:30:35.591+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.11.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 11 (PID: 44901) UID: 0 2026-02-22T23:30:35.597 INFO:tasks.ceph.osd.11.trial189.stderr:2026-02-22T23:30:35.592+0000 beae640 -1 osd.11 48 *** Got signal Terminated *** 2026-02-22T23:30:47.791 INFO:tasks.ceph.osd.11:Stopped 2026-02-22T23:30:47.791 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2026-02-22T23:30:47.791 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:47.799 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-22T23:30:47.795+0000 beae640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.15.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 15 (PID: 44903) UID: 0 2026-02-22T23:30:47.799 INFO:tasks.ceph.osd.15.trial189.stderr:2026-02-22T23:30:47.796+0000 beae640 -1 osd.15 48 *** Got signal Terminated *** 2026-02-22T23:30:53.619 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:09:37.869 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.620 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:09:37.870 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.629 INFO:tasks.ceph.mon.a.trial006.stderr:==00:00:09:37.879 44699== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.644 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:09:37.856 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.645 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:09:37.891 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.646 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:09:37.857 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.646 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:09:37.893 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.654 INFO:tasks.ceph.mon.c.trial080.stderr:==00:00:09:37.865 44468== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.654 INFO:tasks.ceph.mon.b.trial040.stderr:==00:00:09:37.901 44392== Warning: unimplemented fcntl command: 1036 2026-02-22T23:30:53.893 INFO:tasks.ceph.osd.15:Stopped 2026-02-22T23:30:53.893 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-22T23:30:53.894 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2026-02-22T23:30:53.894 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:53.924 INFO:tasks.ceph.mgr.y:Stopped 2026-02-22T23:30:53.925 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-22T23:30:53.925 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:53.925 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:30:53.926+0000 7f73714b8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 43832) UID: 0 2026-02-22T23:30:53.926 INFO:tasks.ceph.mgr.x.trial189.stderr:2026-02-22T23:30:53.926+0000 7f73714b8640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-22T23:30:53.961 INFO:tasks.ceph.mgr.x:Stopped 2026-02-22T23:30:53.961 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-22T23:30:53.961 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-22T23:30:53.961 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:30:53.972 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-22T23:30:53.966+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 44697) UID: 0 2026-02-22T23:30:53.972 INFO:tasks.ceph.mon.a.trial006.stderr:2026-02-22T23:30:53.966+0000 aca6640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-22T23:30:59.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2026-02-22T23:31:00.063 INFO:tasks.ceph.mon.a:Stopped 2026-02-22T23:31:00.063 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-22T23:31:00.063 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:31:00.071 INFO:tasks.ceph.mon.b.trial040.stderr:2026-02-22T23:31:00.063+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 44390) UID: 0 2026-02-22T23:31:00.072 INFO:tasks.ceph.mon.b.trial040.stderr:2026-02-22T23:31:00.063+0000 aca6640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2026-02-22T23:31:04.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2026-02-22T23:31:04.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2026-02-22T23:31:06.165 INFO:tasks.ceph.mon.b:Stopped 2026-02-22T23:31:06.165 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-22T23:31:06.166 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-22T23:31:06.174 INFO:tasks.ceph.mon.c.trial080.stderr:2026-02-22T23:31:06.166+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 44466) UID: 0 2026-02-22T23:31:06.174 INFO:tasks.ceph.mon.c.trial080.stderr:2026-02-22T23:31:06.167+0000 aca6640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2026-02-22T23:31:09.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2026-02-22T23:31:09.682 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2026-02-22T23:31:12.267 INFO:tasks.ceph.mon.c:Stopped 2026-02-22T23:31:12.268 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-22T23:31:12.268 DEBUG:teuthology.orchestra.run.trial006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-22T23:31:12.297 INFO:teuthology.orchestra.run.trial006.stdout:2026-02-22T23:23:21.871240+0000 mon.a (mon.0) 186 : cluster [WRN] Health check failed: 8 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-22T23:31:12.298 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2026-02-22T23:31:12.298 DEBUG:teuthology.orchestra.run.trial006:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-22T23:31:12.363 DEBUG:teuthology.orchestra.run.trial006:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-22T23:31:12.430 DEBUG:teuthology.orchestra.run.trial006:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'reached quota' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(OBJECT_MISPLACED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v 'OSD bench result' | head -n 1 2026-02-22T23:31:12.495 INFO:teuthology.orchestra.run.trial006.stdout:2026-02-22T23:23:21.871240+0000 mon.a (mon.0) 186 : cluster [WRN] Health check failed: 8 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-22T23:31:12.496 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:31:12.496 DEBUG:teuthology.orchestra.run.trial006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-22T23:31:12.569 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:31:12.569 DEBUG:teuthology.orchestra.run.trial006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2026-02-22T23:31:12.638 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:31:12.638 DEBUG:teuthology.orchestra.run.trial006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2026-02-22T23:31:12.706 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:31:12.706 DEBUG:teuthology.orchestra.run.trial006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2026-02-22T23:31:12.774 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:31:12.774 DEBUG:teuthology.orchestra.run.trial040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-22T23:31:12.808 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:31:12.809 DEBUG:teuthology.orchestra.run.trial040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2026-02-22T23:31:12.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:31:12.875 DEBUG:teuthology.orchestra.run.trial040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2026-02-22T23:31:12.943 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:31:12.943 DEBUG:teuthology.orchestra.run.trial040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2026-02-22T23:31:13.011 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:31:13.012 DEBUG:teuthology.orchestra.run.trial080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-22T23:31:13.051 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:31:13.051 DEBUG:teuthology.orchestra.run.trial080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2026-02-22T23:31:13.118 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:31:13.118 DEBUG:teuthology.orchestra.run.trial080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2026-02-22T23:31:13.185 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:31:13.185 DEBUG:teuthology.orchestra.run.trial080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2026-02-22T23:31:13.253 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:31:13.253 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-22T23:31:13.286 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:31:13.286 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2026-02-22T23:31:13.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:31:13.405 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2026-02-22T23:31:13.439 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:31:13.439 DEBUG:teuthology.orchestra.run.trial189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2026-02-22T23:31:13.513 INFO:tasks.ceph:Archiving mon data... 2026-02-22T23:31:13.514 DEBUG:teuthology.misc:Transferring archived files from trial006:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/data/mon.a.tgz 2026-02-22T23:31:13.515 DEBUG:teuthology.orchestra.run.trial006:> mktemp 2026-02-22T23:31:13.532 INFO:teuthology.orchestra.run.trial006.stdout:/tmp/tmp.WnLDli989m 2026-02-22T23:31:13.533 DEBUG:teuthology.orchestra.run.trial006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WnLDli989m 2026-02-22T23:31:13.644 DEBUG:teuthology.orchestra.run.trial006:> sudo chmod 0666 /tmp/tmp.WnLDli989m 2026-02-22T23:31:13.725 DEBUG:teuthology.orchestra.remote:trial006:/tmp/tmp.WnLDli989m is 413KB 2026-02-22T23:31:13.824 DEBUG:teuthology.orchestra.run.trial006:> rm -fr /tmp/tmp.WnLDli989m 2026-02-22T23:31:13.841 DEBUG:teuthology.misc:Transferring archived files from trial040:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/data/mon.b.tgz 2026-02-22T23:31:13.842 DEBUG:teuthology.orchestra.run.trial040:> mktemp 2026-02-22T23:31:13.860 INFO:teuthology.orchestra.run.trial040.stdout:/tmp/tmp.vnrkk7u8i0 2026-02-22T23:31:13.860 DEBUG:teuthology.orchestra.run.trial040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vnrkk7u8i0 2026-02-22T23:31:13.972 DEBUG:teuthology.orchestra.run.trial040:> sudo chmod 0666 /tmp/tmp.vnrkk7u8i0 2026-02-22T23:31:14.052 DEBUG:teuthology.orchestra.remote:trial040:/tmp/tmp.vnrkk7u8i0 is 413KB 2026-02-22T23:31:14.147 DEBUG:teuthology.orchestra.run.trial040:> rm -fr /tmp/tmp.vnrkk7u8i0 2026-02-22T23:31:14.164 DEBUG:teuthology.misc:Transferring archived files from trial080:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/data/mon.c.tgz 2026-02-22T23:31:14.164 DEBUG:teuthology.orchestra.run.trial080:> mktemp 2026-02-22T23:31:14.182 INFO:teuthology.orchestra.run.trial080.stdout:/tmp/tmp.X3Ln3VzxJs 2026-02-22T23:31:14.183 DEBUG:teuthology.orchestra.run.trial080:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.X3Ln3VzxJs 2026-02-22T23:31:14.293 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 0666 /tmp/tmp.X3Ln3VzxJs 2026-02-22T23:31:14.374 DEBUG:teuthology.orchestra.remote:trial080:/tmp/tmp.X3Ln3VzxJs is 413KB 2026-02-22T23:31:14.468 DEBUG:teuthology.orchestra.run.trial080:> rm -fr /tmp/tmp.X3Ln3VzxJs 2026-02-22T23:31:14.485 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-22T23:31:14.485 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T23:31:14.488 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T23:31:14.491 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T23:31:14.527 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T23:31:14.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2026-02-22T23:31:14.819 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-22T23:31:15.559 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-22T23:31:15.559 INFO:tasks.ceph:Archiving crash dumps... 2026-02-22T23:31:15.562 DEBUG:teuthology.misc:Transferring archived files from trial006:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial006/crash 2026-02-22T23:31:15.563 DEBUG:teuthology.orchestra.run.trial006:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T23:31:15.589 DEBUG:teuthology.misc:Transferring archived files from trial040:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial040/crash 2026-02-22T23:31:15.591 DEBUG:teuthology.orchestra.run.trial040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T23:31:15.617 DEBUG:teuthology.misc:Transferring archived files from trial080:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial080/crash 2026-02-22T23:31:15.619 DEBUG:teuthology.orchestra.run.trial080:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T23:31:15.646 DEBUG:teuthology.misc:Transferring archived files from trial189:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial189/crash 2026-02-22T23:31:15.647 DEBUG:teuthology.orchestra.run.trial189:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T23:31:15.674 INFO:tasks.ceph:Compressing logs... 2026-02-22T23:31:15.674 DEBUG:teuthology.orchestra.run.trial006:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:15.678 DEBUG:teuthology.orchestra.run.trial040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:15.680 DEBUG:teuthology.orchestra.run.trial080:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:15.691 DEBUG:teuthology.orchestra.run.trial189:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:15.700 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55948.log 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50336.log 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58788.log 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55948.log.gz/var/log/ceph/ceph-client.admin.50336.log: 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51192.log 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58788.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50336.log.gz 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51192.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58788.log.gz 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57712.log 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51192.log.gz 2026-02-22T23:31:15.703 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57565.log 2026-02-22T23:31:15.704 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52500.log/var/log/ceph/ceph-client.admin.57712.log: 2026-02-22T23:31:15.704 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51762.log 2026-02-22T23:31:15.704 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57712.log.gz 2026-02-22T23:31:15.704 INFO:teuthology.orchestra.run.trial006.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2026-02-22T23:31:15.704 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52500.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-22T23:31:15.704 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51528.log 2026-02-22T23:31:15.712 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-22T23:31:15.712 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.5.log 2026-02-22T23:31:15.712 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.13.log 2026-02-22T23:31:15.712 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/valgrind/osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.1.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.5.log.gz 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/valgrind/osd.13.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.9.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr: 67.8% -- replaced with /var/log/ceph/valgrind/osd.13.log.gz 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/valgrind/osd.1.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/valgrind/osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.9.log.gz 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/ceph-mon.b.log: 94.1% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/ceph-osd.13.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-22T23:31:15.713 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2026-02-22T23:31:15.714 INFO:teuthology.orchestra.run.trial040.stderr: 90.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-22T23:31:15.714 INFO:teuthology.orchestra.run.trial040.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-osd.9.log: 89.9% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-22T23:31:15.720 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-22T23:31:15.720 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-22T23:31:15.720 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.2.log 2026-02-22T23:31:15.720 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.6.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.14.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/valgrind/osd.2.log: 67.9% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/valgrind/osd.6.log: 67.9% -- replaced with /var/log/ceph/valgrind/osd.6.log.gz 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.10.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose/var/log/ceph/valgrind/osd.14.log: -- /var/log/ceph/valgrind/mon.c.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr: 68.3% -- replaced with /var/log/ceph/valgrind/osd.14.log.gz 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/valgrind/osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.10.log.gz 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/valgrind/mon.c.log: /var/log/ceph/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/ceph-osd.14.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2026-02-22T23:31:15.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/ceph.audit.log: 94.1% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2026-02-22T23:31:15.722 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-22T23:31:15.722 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph.log: 90.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-22T23:31:15.722 INFO:teuthology.orchestra.run.trial080.stderr: 89.9% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44449.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44590.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44449.log.gz 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44590.log: /var/log/ceph/ceph-client.admin.44402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44355.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44590.log.gz 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44731.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-osd.3.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44355.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44355.log.gz 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44731.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.7.log 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44731.log.gz 2026-02-22T23:31:15.750 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.11.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/valgrind/osd.7.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.15.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.7.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/valgrind/osd.11.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.3.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 68.1% -- replaced with /var/log/ceph/valgrind/osd.11.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/valgrind/osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44134.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.15.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/valgrind/osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 67.8% -- replaced with /var/log/ceph/valgrind/osd.3.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44283.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-osd.11.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44134.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44778.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44637.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44778.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44283.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44181.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5/var/log/ceph/ceph-client.admin.44637.log: --verbose -- /var/log/ceph/ceph-client.admin.44496.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44778.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44543.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44637.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44181.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44684.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43982.log 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44496.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44684.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44543.log.gz 2026-02-22T23:31:15.751 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44087.log 2026-02-22T23:31:15.752 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.43982.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-22T23:31:15.752 INFO:teuthology.orchestra.run.trial189.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44684.log.gz 2026-02-22T23:31:15.752 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44087.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43982.log.gz 2026-02-22T23:31:15.752 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44031.log 2026-02-22T23:31:15.752 INFO:teuthology.orchestra.run.trial189.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44087.log.gz 2026-02-22T23:31:15.752 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2026-02-22T23:31:15.752 INFO:teuthology.orchestra.run.trial189.stderr:/var/log/ceph/ceph-client.admin.44031.log: /var/log/ceph/ceph-osd.15.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44031.log.gz 2026-02-22T23:31:15.758 INFO:teuthology.orchestra.run.trial040.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-22T23:31:15.759 INFO:teuthology.orchestra.run.trial040.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-osd.0.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52500.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51762.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52892.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51528.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51528.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54917.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-mgr.y.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.52892.log.gz -5 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51431.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57614.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53480.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51431.log: /var/log/ceph/ceph-client.admin.57614.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54917.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52794.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51431.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53480.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57614.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52401.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52794.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53480.log.gz 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51881.log 2026-02-22T23:31:15.764 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52401.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52794.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50311.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52401.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52990.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54280.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52990.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50311.log.gzgzip 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56291.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52990.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54378.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56291.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54280.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56144.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49755.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56291.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56144.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54378.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52647.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49755.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56144.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57124.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52647.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49755.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51931.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52647.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57173.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57124.log: /var/log/ceph/ceph-client.admin.51931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55064.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 87.7%/var/log/ceph/ceph-client.admin.57173.log: -- replaced with /var/log/ceph/ceph-client.admin.57124.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52598.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51931.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55064.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57173.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55407.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52598.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55064.log.gz 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46289.log 2026-02-22T23:31:15.765 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55407.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52598.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57222.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55407.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53578.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46289.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49747.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57222.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53578.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52303.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53578.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49718.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52303.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49747.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54131.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53186.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52303.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55702.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49718.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54131.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55850.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53836.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53186.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55850.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55702.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51335.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53836.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55850.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52352.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53836.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58104.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50348.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51335.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52352.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53983.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49752.log 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58104.log.gz 2026-02-22T23:31:15.766 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50348.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48790.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49737.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53983.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48790.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49752.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56242.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48790.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45080.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56242.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49737.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58300.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50927.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56242.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58300.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45080.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50927.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.12.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58300.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50927.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/valgrind/osd.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.8.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/valgrind/osd.12.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.12.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/valgrind/osd.8.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.4.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/valgrind/mon.a.log: 67.8% -- replaced with /var/log/ceph/valgrind/osd.8.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57026.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/valgrind/osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49758.log 94.1% 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.4.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49306.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57467.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57026.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49758.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48606.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57467.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49306.log.gz 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51715.log 2026-02-22T23:31:15.767 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48606.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57467.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58496.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48606.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56732.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50652.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51715.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52016.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58496.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56732.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56928.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52016.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50652.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54032.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52016.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54476.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54032.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56928.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49257.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54476.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54032.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49257.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54476.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49723.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49257.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56585.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49723.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49723.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58202.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45317.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56585.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52203.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58202.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56193.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52203.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58202.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51217.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52203.log.gz 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56634.log 2026-02-22T23:31:15.768 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56193.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50493.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51217.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49759.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56781.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56634.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57516.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50493.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56781.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49759.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49760.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56781.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48698.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55555.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57516.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48698.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49760.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51836.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55555.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48698.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53885.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51836.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55555.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55113.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51836.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54623.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56046.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53885.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53333.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55113.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56046.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57761.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56046.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51450.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57761.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53333.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56340.log 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57761.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51450.log.gz 2026-02-22T23:31:15.769 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54329.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58741.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56340.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54329.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz --verbose 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: -- /var/log/ceph/ceph-client.admin.53676.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58741.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54329.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50956.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53627.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58741.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58153.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53676.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53627.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50956.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49131.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53627.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52082.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58153.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57271.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57957.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52082.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57271.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49131.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51584.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52549.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57271.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52745.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57957.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51584.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49108.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52549.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54770.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51578.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52745.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54770.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49108.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50338.log 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54770.log.gz 2026-02-22T23:31:15.770 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55358.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53039.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51578.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50338.log.gz/var/log/ceph/ceph-client.admin.55358.log: 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58349.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55358.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55604.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52014.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53039.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55604.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58349.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50505.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55604.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58692.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50505.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52014.log.gz /var/log/ceph/ceph.audit.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50505.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57810.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58692.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58545.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57810.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57320.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 90.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53137.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58594.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58545.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53137.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57320.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49726.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53137.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53431.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49726.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58594.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54231.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53431.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49726.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45149.log 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54231.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53431.log.gz 2026-02-22T23:31:15.771 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51021.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.45149.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54231.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56830.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51021.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45149.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48744.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56830.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51021.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56977.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48744.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56830.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48744.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44750.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53235.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56977.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 88.0%/var/log/ceph/ceph-client.admin.44750.log: -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57369.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58006.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57369.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53235.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.44750.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55899.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57369.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54819.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50347.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58006.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54819.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55899.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51451.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50347.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54819.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55162.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50347.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50653.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54966.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51451.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57663.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55162.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54966.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50653.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58251.log 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-22T23:31:15.772 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54966.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54525.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57663.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57418.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57075.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58251.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56683.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54525.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56487.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57075.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57418.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52450.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57075.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56487.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56683.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51092.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56487.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52696.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55751.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51092.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52450.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip/var/log/ceph/ceph-client.admin.52696.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49743.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55751.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51092.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53774.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52696.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55751.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54181.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip/var/log/ceph/ceph-client.admin.53774.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49743.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55015.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53774.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49746.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54181.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58055.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 87.7%/var/log/ceph/ceph-client.admin.55015.log: -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55801.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58055.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55015.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53529.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58055.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52941.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55801.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53529.log: /var/log/ceph/ceph-client.admin.57859.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52941.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55801.log.gz 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56536.log 2026-02-22T23:31:15.773 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53529.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52941.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58643.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55997.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57859.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56536.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55309.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50528.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55309.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58643.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55997.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51093.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55309.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51218.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48475.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51093.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50528.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51093.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48475.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51218.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49729.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48475.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54721.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.52248.log: /var/log/ceph/ceph-client.admin.49729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51790.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54721.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49729.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52149.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54721.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51790.log: /var/log/ceph/ceph-client.admin.52149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54427.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51790.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.46134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51306.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52149.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55457.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54427.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.51306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54427.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54868.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55457.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51306.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56389.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55457.log.gz 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54081.log 2026-02-22T23:31:15.774 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53088.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54868.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54081.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.56389.log.gz --verbose 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: -- /var/log/ceph/ceph-client.admin.56438.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53088.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54081.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53725.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56438.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56879.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55211.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56879.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53725.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 89.9% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55211.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56879.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50388.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-osd.8.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55211.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55260.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58398.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55260.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50388.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58447.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58398.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55260.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53284.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.58447.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58398.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55653.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58447.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50314.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53284.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55653.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53934.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50314.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55653.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54574.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57908.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53934.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50314.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50492.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53934.log.gz 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.57908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49757.log 2026-02-22T23:31:15.775 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54574.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.50492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49741.log 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57908.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56095.log 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50492.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.49741.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38399.log 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49757.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.56095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54672.log 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49741.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph.tmp-client.admin.38399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53382.log 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38399.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56095.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.54672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55506.log 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.53382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48652.log 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54672.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.55506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53382.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr:/var/log/ceph/ceph-client.admin.48652.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55506.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial006.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48652.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial040.stderr: 90.6% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial080.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial080.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2026-02-22T23:31:15.776 INFO:teuthology.orchestra.run.trial080.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2026-02-22T23:31:15.778 INFO:teuthology.orchestra.run.trial006.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2026-02-22T23:31:15.781 INFO:teuthology.orchestra.run.trial080.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2026-02-22T23:31:15.790 INFO:teuthology.orchestra.run.trial189.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-22T23:31:15.793 INFO:teuthology.orchestra.run.trial006.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-22T23:31:15.798 INFO:teuthology.orchestra.run.trial189.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-22T23:31:15.802 INFO:teuthology.orchestra.run.trial189.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2026-02-22T23:31:15.804 INFO:teuthology.orchestra.run.trial189.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2026-02-22T23:31:15.806 INFO:teuthology.orchestra.run.trial040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-22T23:31:15.807 INFO:teuthology.orchestra.run.trial040.stderr: 2026-02-22T23:31:15.807 INFO:teuthology.orchestra.run.trial040.stderr:real 0m0.113s 2026-02-22T23:31:15.807 INFO:teuthology.orchestra.run.trial040.stderr:user 0m0.316s 2026-02-22T23:31:15.807 INFO:teuthology.orchestra.run.trial040.stderr:sys 0m0.038s 2026-02-22T23:31:15.808 INFO:teuthology.orchestra.run.trial189.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2026-02-22T23:31:15.809 INFO:teuthology.orchestra.run.trial006.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2026-02-22T23:31:15.809 INFO:teuthology.orchestra.run.trial189.stderr: 2026-02-22T23:31:15.809 INFO:teuthology.orchestra.run.trial189.stderr:real 0m0.077s 2026-02-22T23:31:15.809 INFO:teuthology.orchestra.run.trial189.stderr:user 0m0.278s 2026-02-22T23:31:15.809 INFO:teuthology.orchestra.run.trial189.stderr:sys 0m0.043s 2026-02-22T23:31:15.812 INFO:teuthology.orchestra.run.trial080.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-22T23:31:15.814 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-22T23:31:15.814 INFO:teuthology.orchestra.run.trial080.stderr:real 0m0.111s 2026-02-22T23:31:15.814 INFO:teuthology.orchestra.run.trial080.stderr:user 0m0.329s 2026-02-22T23:31:15.814 INFO:teuthology.orchestra.run.trial080.stderr:sys 0m0.036s 2026-02-22T23:31:15.890 INFO:teuthology.orchestra.run.trial006.stderr: 89.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-22T23:31:15.891 INFO:teuthology.orchestra.run.trial006.stderr: 2026-02-22T23:31:15.891 INFO:teuthology.orchestra.run.trial006.stderr:real 0m0.200s 2026-02-22T23:31:15.892 INFO:teuthology.orchestra.run.trial006.stderr:user 0m0.470s 2026-02-22T23:31:15.892 INFO:teuthology.orchestra.run.trial006.stderr:sys 0m0.142s 2026-02-22T23:31:15.893 INFO:tasks.ceph:Archiving logs... 2026-02-22T23:31:15.893 DEBUG:teuthology.misc:Transferring archived files from trial006:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial006/log 2026-02-22T23:31:15.894 DEBUG:teuthology.orchestra.run.trial006:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T23:31:16.213 DEBUG:teuthology.misc:Transferring archived files from trial040:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial040/log 2026-02-22T23:31:16.213 DEBUG:teuthology.orchestra.run.trial040:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T23:31:16.328 DEBUG:teuthology.misc:Transferring archived files from trial080:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial080/log 2026-02-22T23:31:16.329 DEBUG:teuthology.orchestra.run.trial080:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T23:31:16.403 DEBUG:teuthology.misc:Transferring archived files from trial189:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial189/log 2026-02-22T23:31:16.404 DEBUG:teuthology.orchestra.run.trial189:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T23:31:16.488 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1919, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 46, in nested if exit(*exc): ^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-22T23:31:16.489 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-22T23:31:16.515 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-22T23:31:16.532 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-22T23:31:16.532 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T23:31:16.535 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T23:31:16.537 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T23:31:16.539 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T23:31:16.814 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-22T23:31:16.825 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-22T23:31:16.825 DEBUG:teuthology.orchestra.run.trial006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:31:16.827 DEBUG:teuthology.orchestra.run.trial040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:31:16.829 DEBUG:teuthology.orchestra.run.trial080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:31:16.831 DEBUG:teuthology.orchestra.run.trial189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:31:16.841 INFO:teuthology.orchestra.run.trial006.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:31:16.843 INFO:teuthology.orchestra.run.trial040.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:31:16.845 INFO:teuthology.orchestra.run.trial080.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:31:16.848 INFO:teuthology.orchestra.run.trial189.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:31:16.867 INFO:teuthology.orchestra.run.trial040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:31:16.868 INFO:teuthology.orchestra.run.trial040.stdout:=============================================================================== 2026-02-22T23:31:16.868 INFO:teuthology.orchestra.run.trial040.stdout:^+ time.ecansol.net 1 7 377 45 +2307us[+2058us] +/- 16ms 2026-02-22T23:31:16.868 INFO:teuthology.orchestra.run.trial040.stdout:^* 38.70.62.230 1 6 377 44 -1004us[-1254us] +/- 6819us 2026-02-22T23:31:16.868 INFO:teuthology.orchestra.run.trial040.stdout:^- 172-234-25-10.ip.linodeu> 2 6 377 45 -8109us[-8358us] +/- 75ms 2026-02-22T23:31:16.868 INFO:teuthology.orchestra.run.trial040.stdout:^- 155.248.196.28 2 6 377 42 -1956us[-1956us] +/- 50ms 2026-02-22T23:31:16.874 INFO:teuthology.orchestra.run.trial006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:31:16.874 INFO:teuthology.orchestra.run.trial006.stdout:=============================================================================== 2026-02-22T23:31:16.874 INFO:teuthology.orchestra.run.trial006.stdout:^- nc7j.com 1 6 377 47 -790us[ -768us] +/- 28ms 2026-02-22T23:31:16.874 INFO:teuthology.orchestra.run.trial006.stdout:^- 66.42.71.197.vultruserco> 2 6 377 43 -347us[ -347us] +/- 47ms 2026-02-22T23:31:16.874 INFO:teuthology.orchestra.run.trial006.stdout:^* time.cloudflare.com 3 6 377 46 +69us[ +91us] +/- 6462us 2026-02-22T23:31:16.874 INFO:teuthology.orchestra.run.trial006.stdout:^- ntp6.kernfusion.at 2 6 377 47 +262us[ +284us] +/- 47ms 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial189.stdout:=============================================================================== 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial189.stdout:^* ntp.sfo.icanbwell.com 2 7 377 47 +1994us[+2117us] +/- 35ms 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial189.stdout:^- 172-233-157-223.ip.linod> 2 6 377 177 -7870us[-7625us] +/- 122ms 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial189.stdout:^+ 139-177-202-26.ip.linode> 2 7 377 48 -7571us[-7447us] +/- 79ms 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial189.stdout:^+ gwallaceNTPpool.com 2 6 377 45 +2780us[+2780us] +/- 58ms 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial080.stdout:=============================================================================== 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial080.stdout:^- 99-28-14-242.lightspeed.> 1 6 377 43 +236us[ +236us] +/- 13ms 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial080.stdout:^* ns3.smatwebdesign.com 2 6 377 45 -98us[ -127us] +/- 4762us 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial080.stdout:^- ip229.ip-51-81-226.us 2 7 377 45 +835us[ +835us] +/- 122ms 2026-02-22T23:31:16.879 INFO:teuthology.orchestra.run.trial080.stdout:^- us-east-2.clearnet.pw 2 7 377 44 -1296us[-1296us] +/- 61ms 2026-02-22T23:31:16.880 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-22T23:31:16.892 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-22T23:31:16.892 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-22T23:31:16.905 DEBUG:teuthology.orchestra.run.trial006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-22T23:31:16.973 DEBUG:teuthology.orchestra.run.trial040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-22T23:31:17.027 DEBUG:teuthology.orchestra.run.trial080:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-22T23:31:17.082 DEBUG:teuthology.orchestra.run.trial189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-22T23:31:17.139 DEBUG:teuthology.orchestra.run.trial006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:31:17.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:17.167 DEBUG:teuthology.orchestra.run.trial040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:31:17.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:17.196 DEBUG:teuthology.orchestra.run.trial080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:31:17.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:17.225 DEBUG:teuthology.orchestra.run.trial189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:31:17.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:17.253 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-22T23:31:17.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-22T23:31:17.286 INFO:teuthology.task.internal:Duration was 1189.080200 seconds 2026-02-22T23:31:17.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-22T23:31:17.300 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-22T23:31:17.300 DEBUG:teuthology.orchestra.run.trial006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T23:31:17.303 DEBUG:teuthology.orchestra.run.trial040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T23:31:17.306 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T23:31:17.308 DEBUG:teuthology.orchestra.run.trial189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T23:31:17.339 INFO:teuthology.orchestra.run.trial006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:31:17.341 INFO:teuthology.orchestra.run.trial040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:31:17.343 INFO:teuthology.orchestra.run.trial080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:31:17.345 INFO:teuthology.orchestra.run.trial189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:31:17.701 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-22T23:31:17.701 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial006.front.sepia.ceph.com 2026-02-22T23:31:17.701 DEBUG:teuthology.orchestra.run.trial006:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T23:31:17.731 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial040.front.sepia.ceph.com 2026-02-22T23:31:17.731 DEBUG:teuthology.orchestra.run.trial040:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T23:31:17.761 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial080.front.sepia.ceph.com 2026-02-22T23:31:17.762 DEBUG:teuthology.orchestra.run.trial080:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T23:31:17.782 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial189.front.sepia.ceph.com 2026-02-22T23:31:17.782 DEBUG:teuthology.orchestra.run.trial189:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T23:31:17.803 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-22T23:31:17.803 DEBUG:teuthology.orchestra.run.trial006:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.806 DEBUG:teuthology.orchestra.run.trial040:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.809 DEBUG:teuthology.orchestra.run.trial080:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.824 DEBUG:teuthology.orchestra.run.trial189:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.905 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-22T23:31:17.905 DEBUG:teuthology.orchestra.run.trial006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:17.907 DEBUG:teuthology.orchestra.run.trial040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:17.910 DEBUG:teuthology.orchestra.run.trial080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:17.912 DEBUG:teuthology.orchestra.run.trial189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:31:17.928 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:31:17.929 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:31:17.929 INFO:teuthology.orchestra.run.trial006.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T23:31:17.929 INFO:teuthology.orchestra.run.trial006.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.929 INFO:teuthology.orchestra.run.trial006.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T23:31:17.930 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:31:17.931 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:31:17.931 INFO:teuthology.orchestra.run.trial040.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T23:31:17.931 INFO:teuthology.orchestra.run.trial040.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.931 INFO:teuthology.orchestra.run.trial040.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T23:31:17.933 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:31:17.934 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:31:17.934 INFO:teuthology.orchestra.run.trial080.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.934 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T23:31:17.935 INFO:teuthology.orchestra.run.trial080.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T23:31:17.936 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:31:17.937 INFO:teuthology.orchestra.run.trial189.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:31:17.937 INFO:teuthology.orchestra.run.trial189.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:31:17.937 INFO:teuthology.orchestra.run.trial189.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T23:31:17.937 INFO:teuthology.orchestra.run.trial189.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T23:31:17.940 INFO:teuthology.orchestra.run.trial006.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T23:31:17.942 INFO:teuthology.orchestra.run.trial040.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T23:31:17.944 INFO:teuthology.orchestra.run.trial080.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T23:31:17.946 INFO:teuthology.orchestra.run.trial189.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T23:31:17.948 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-22T23:31:17.963 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-22T23:31:17.963 DEBUG:teuthology.orchestra.run.trial006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T23:31:18.006 DEBUG:teuthology.orchestra.run.trial040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T23:31:18.030 DEBUG:teuthology.orchestra.run.trial080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T23:31:18.057 DEBUG:teuthology.orchestra.run.trial189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T23:31:18.083 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-22T23:31:18.098 DEBUG:teuthology.orchestra.run.trial006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.101 DEBUG:teuthology.orchestra.run.trial040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.103 DEBUG:teuthology.orchestra.run.trial080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.106 DEBUG:teuthology.orchestra.run.trial189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.123 INFO:teuthology.orchestra.run.trial006.stdout:kernel.core_pattern = core 2026-02-22T23:31:18.126 INFO:teuthology.orchestra.run.trial040.stdout:kernel.core_pattern = core 2026-02-22T23:31:18.128 INFO:teuthology.orchestra.run.trial080.stdout:kernel.core_pattern = core 2026-02-22T23:31:18.147 INFO:teuthology.orchestra.run.trial189.stdout:kernel.core_pattern = core 2026-02-22T23:31:18.161 DEBUG:teuthology.orchestra.run.trial006:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:18.197 DEBUG:teuthology.orchestra.run.trial040:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:18.213 DEBUG:teuthology.orchestra.run.trial080:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:18.231 DEBUG:teuthology.orchestra.run.trial189:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:31:18.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:31:18.247 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-22T23:31:18.258 INFO:teuthology.task.internal:Transferring archived files... 2026-02-22T23:31:18.259 DEBUG:teuthology.misc:Transferring archived files from trial006:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial006 2026-02-22T23:31:18.259 DEBUG:teuthology.orchestra.run.trial006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T23:31:18.295 DEBUG:teuthology.misc:Transferring archived files from trial040:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial040 2026-02-22T23:31:18.295 DEBUG:teuthology.orchestra.run.trial040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T23:31:18.332 DEBUG:teuthology.misc:Transferring archived files from trial080:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial080 2026-02-22T23:31:18.332 DEBUG:teuthology.orchestra.run.trial080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T23:31:18.369 DEBUG:teuthology.misc:Transferring archived files from trial189:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64361/remote/trial189 2026-02-22T23:31:18.370 DEBUG:teuthology.orchestra.run.trial189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T23:31:18.407 INFO:teuthology.task.internal:Removing archive directory... 2026-02-22T23:31:18.407 DEBUG:teuthology.orchestra.run.trial006:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T23:31:18.410 DEBUG:teuthology.orchestra.run.trial040:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T23:31:18.413 DEBUG:teuthology.orchestra.run.trial080:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T23:31:18.416 DEBUG:teuthology.orchestra.run.trial189:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T23:31:18.456 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-22T23:31:18.476 INFO:teuthology.task.internal:Not uploading archives. 2026-02-22T23:31:18.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-22T23:31:18.492 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-22T23:31:18.492 DEBUG:teuthology.orchestra.run.trial006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T23:31:18.495 DEBUG:teuthology.orchestra.run.trial040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T23:31:18.498 DEBUG:teuthology.orchestra.run.trial080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T23:31:18.500 DEBUG:teuthology.orchestra.run.trial189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T23:31:18.510 INFO:teuthology.orchestra.run.trial006.stdout: 917521 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 23:31 /home/ubuntu/cephtest 2026-02-22T23:31:18.513 INFO:teuthology.orchestra.run.trial040.stdout: 917572 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 23:31 /home/ubuntu/cephtest 2026-02-22T23:31:18.515 INFO:teuthology.orchestra.run.trial080.stdout: 917572 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 23:31 /home/ubuntu/cephtest 2026-02-22T23:31:18.516 INFO:teuthology.orchestra.run.trial189.stdout: 917521 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 23:31 /home/ubuntu/cephtest 2026-02-22T23:31:18.517 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-22T23:31:18.534 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-22T23:31:18.570 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/fixed-4 d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} duration: 1189.080199956894 failure_reason: '"2026-02-22T23:23:21.871240+0000 mon.a (mon.0) 186 : cluster [WRN] Health check failed: 8 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT)" in cluster log' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-22T23:31:18.570 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T23:31:18.698 INFO:teuthology.run:FAIL